Imaging apparatus, image data processing method of imaging apparatus, and program

ABSTRACT

An imaging apparatus includes a storage portion that stores captured image data obtained by imaging a subject by an imaging element and is incorporated in the imaging element, an output portion that is incorporated in the imaging element, and a plurality of signal processing portions that are disposed outside the imaging element, in which the output portion includes a plurality of output lines each disposed in correspondence with each of the plurality of signal processing portions and outputs each of a plurality of pieces of image data into which the captured image data stored in the storage portion is divided, to a corresponding signal processing portion among the plurality of signal processing portions from the plurality of output lines, and any of the plurality of signal processing portions combines the plurality of pieces of image data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. Ser. No.17/351,273, filed on Jun. 18, 2021, which is a continuation applicationof International Application No. PCT/JP2019/049218, filed on Dec. 16,2019. Further, this application claims priority from Japanese PatentApplication No. 2018-243662, filed on Dec. 26, 2018. The disclosure ofeach of the above applications is incorporated herein by reference inits entirety.

BACKGROUND 1. Technical Field

The technology of the present disclosure relates to an imagingapparatus, an image data processing method of an imaging apparatus, anda program.

2. Related Art

JP2016-158294A discloses an electronic apparatus comprising an imagingelement, an image processing portion, and a control portion. The imagingelement has a first imaging region and a second imaging region. In theimaging element, pixels are alternately arranged in the first imagingregion and the second imaging region. In a case of a first imagingcondition, the imaging element performs imaging using the first imagingregion. In a case of a second imaging condition different from the firstimaging condition, the imaging element performs imaging using the secondimaging region. Image data consisting of pixel signals of the firstimaging region and image data consisting of pixel signals of the secondimaging region are output to the image processing portion in a rearstage through one output line.

The image processing portion generates first image data by performingvarious types of image processing on the image data consisting of thepixel signals of the first imaging region, and generates second imagedata by performing various types of image processing on the image dataconsisting of the pixel signals of the second imaging region. Thecontrol portion displays a live view image or a still picture in which afirst image indicated by the first image data and a second imageindicated by the second image data are combined, on a display portion.

SUMMARY

One embodiment according to the technology of the present disclosureprovides an imaging apparatus, an image data processing method of animaging apparatus, and a program capable of implementing high-speedimage processing, compared to a case of outputting image data to aplurality of signal processing portions from an imaging element usingonly one output line.

An imaging apparatus according to a first aspect of the technology ofthe present disclosure is an imaging apparatus including an imagingelement and comprises a storage portion that stores captured image dataobtained by imaging a subject by the imaging element and is incorporatedin the imaging element, an output portion that is incorporated in theimaging element, and a plurality of signal processing portions that aredisposed outside the imaging element, in which the output portionincludes a plurality of output lines each disposed in correspondencewith each of the plurality of signal processing portions and outputseach of a plurality of pieces of image data into which the capturedimage data stored in the storage portion is divided, to a correspondingsignal processing portion among the plurality of signal processingportions from the plurality of output lines, and any of the plurality ofsignal processing portions combines the plurality of pieces of imagedata. Accordingly, high-speed image processing can be implemented,compared to a case of outputting image data to the plurality of signalprocessing portions from the imaging element using only one output line.

In the imaging apparatus according to a second aspect of the technologyof the present disclosure, each of the plurality of pieces of image datais image data indicating an image having an overlapping region betweenadjacent images among images based on each of the plurality of pieces ofimage data. Accordingly, noticeability of a boundary region between twoimages is suppressed, compared to a case of joining two images obtainedby simply dividing a captured image into two parts.

In the imaging apparatus according to a third aspect of the technologyof the present disclosure, the plurality of images are divided into adesignated image and an image different from the designated image.Accordingly, noticeability of a boundary region between the designatedimage and the image different from the designated image is suppressed.

The imaging apparatus according to a fourth aspect of the technology ofthe present disclosure further comprises a detection portion thatdetects face image data indicating an image of a face from the capturedimage data, in which the designated image is an image including theimage of the face indicated by the face image data detected by thedetection portion in a captured image indicated by the captured imagedata. Accordingly, noticeability of a boundary region between the imageincluding the image of the face and an image not including the image ofthe face is suppressed.

In the imaging apparatus according to a fifth aspect of the technologyof the present disclosure, a division method for the captured image datavaries between a recording imaging mode and a display motion picturecapturing mode. Accordingly, a balance among image quality, powerconsumption, and a processing speed can be set to vary between therecording imaging mode and the display motion picture capturing mode.

In the imaging apparatus according to a sixth aspect of the technologyof the present disclosure, the captured image data is divided into aplurality of pieces of overlapping image data as the plurality of piecesof image data in the recording imaging mode, and the captured image datais divided in units of lines in the display motion picture capturingmode. Accordingly, in the recording imaging mode, the image quality canbe increased, compared to the display motion picture capturing mode. Inthe display motion picture capturing mode, the power consumption can bereduced, and the processing speed can be increased, compared to therecording imaging mode.

In the imaging apparatus according to a seventh aspect of the technologyof the present disclosure, each of the plurality of pieces ofoverlapping image data is image data indicating an image having anoverlapping region between adjacent images among the plurality ofimages. Accordingly, noticeability of a boundary region between twoimages is suppressed, compared to a case of joining two images obtainedby simply dividing a captured image into two parts.

In the imaging apparatus according to an eighth aspect of the technologyof the present disclosure, the recording imaging mode is an operationmode in which the imaging element performs imaging for a still pictureimage. Accordingly, the balance among the image quality, the powerconsumption, and the processing speed can be set to vary between theoperation mode in which the imaging for the still picture image isperformed, and the display motion picture capturing mode.

In the imaging apparatus according to a ninth aspect of the technologyof the present disclosure, the captured image data is color image dataindicating a color captured image in which a plurality of primary colorpixels are periodically arranged, the color image data is divided into aplurality of pieces of primary color pixel arrangement image data as theplurality of pieces of image data, and each of the plurality of piecesof primary color pixel arrangement image data is image data indicatingan image in which each of the plurality of primary color pixels isperiodically arranged. Accordingly, even in a case where the color imagedata is divided into the plurality of pieces of primary color pixelarrangement image data, demosaicing for the plurality of primary colorpixels can be implemented.

In the imaging apparatus according to a tenth aspect of the technologyof the present disclosure, the plurality of pieces of primary colorpixel arrangement image data are a plurality of pieces of divided imagedata obtained by thinning out and then, dividing the color image data.Accordingly, high-speed processing can be implemented, compared to acase where a plurality of pieces of image data obtained by dividing thecolor image data without thinning are processed by the plurality ofsignal processing portions.

In the imaging apparatus according to an eleventh aspect of thetechnology of the present disclosure, the plurality of pieces of dividedimage data are odd-numbered column image data indicating a set of pixelsof odd-numbered columns and even-numbered column image data indicating aset of pixels of even-numbered columns in thinned image data obtained bythinning out the color image data in units of rows. Accordingly, each ofthe signal processing portions can implement high-speed processing,compared to a case of processing image data obtained by irregulardivision.

In the imaging apparatus according to a twelfth aspect of the technologyof the present disclosure, any of the plurality of signal processingportions performs demosaicing on combined image data obtained bycombining the plurality of pieces of image data. Accordingly, a highimage quality image can be obtained, compared to a case of notperforming the demosaicing.

In the imaging apparatus according to a thirteenth aspect of thetechnology of the present disclosure, the plurality of pieces of imagedata are a plurality of pieces of compressed image data obtained bycompressing the captured image data by dividing the captured image datainto a plurality of bit ranges. Accordingly, each of the signalprocessing portions can implement high-speed processing, compared to acase of processing image data obtained by irregular division.

In the imaging apparatus according to a fourteenth aspect of thetechnology of the present disclosure, the plurality of pieces ofcompressed image data are high-order bit image data and low-order bitimage data in the captured image data. Accordingly, high-accuracyprocessing can be performed on the high-order bit image data, comparedto the low-order bit image data. For the low-order bit image data, thepower consumption can be reduced, and the processing speed can beincreased, compared to the high-order bit image data.

In the imaging apparatus according to a fifteenth aspect of thetechnology of the present disclosure, the imaging element is an imagingelement in which at least a photoelectric conversion element and thestorage portion are formed in one chip. Accordingly, portability of theimaging element can be increased, compared to an imaging element inwhich the photoelectric conversion element and the storage portion arenot formed in one chip.

In the imaging apparatus according to a sixteenth aspect of thetechnology of the present disclosure, the imaging element is a laminatedimaging element in which the photoelectric conversion element islaminated with the storage portion. Accordingly, a load exerted onprocessing between the photoelectric conversion element and the storageportion can be reduced, compared to a case of not laminating thephotoelectric conversion element and the storage portion.

The imaging apparatus according to a seventeenth aspect of thetechnology of the present disclosure further comprises a control portionthat performs a control for displaying an image based on the pluralityof pieces of image data output by the output portion on a displayportion. Accordingly, a user can visually recognize the image based onthe plurality of pieces of image data output by the output portion.

An image data processing method according to an eighteenth aspect of thetechnology of the present disclosure is an image data processing methodof an imaging apparatus including an imaging element, a storage portionthat stores captured image data obtained by imaging a subject by theimaging element and is incorporated in the imaging element, an outputportion that is incorporated in the imaging element, and a plurality ofsignal processing portions that are disposed outside the imagingelement, in which the output portion includes a plurality of outputlines each disposed in correspondence with each of the plurality ofsignal processing portions and outputs each of a plurality of pieces ofimage data into which the captured image data stored in the storageportion is divided, to a corresponding signal processing portion amongthe plurality of signal processing portions from the plurality of outputlines, and any of the plurality of signal processing portions combinesthe plurality of pieces of image data. Accordingly, high-speed imageprocessing can be implemented, compared to a case of outputting imagedata to the plurality of signal processing portions from the imagingelement using only one output line.

A program according to a nineteenth aspect of the technology of thepresent disclosure is a program causing a computer to function as anoutput portion included in an imaging apparatus including an imagingelement, a storage portion that stores captured image data obtained byimaging a subject by the imaging element and is incorporated in theimaging element, the output portion that is incorporated in the imagingelement, and a plurality of signal processing portions that are disposedoutside the imaging element, in which the output portion includes aplurality of output lines each disposed in correspondence with each ofthe plurality of signal processing portions and outputs each of aplurality of pieces of image data into which the captured image datastored in the storage portion is divided, to a corresponding signalprocessing portion among the plurality of signal processing portionsfrom the plurality of output lines, and any of the plurality of signalprocessing portions combines the plurality of pieces of image data.Accordingly, high-speed image processing can be implemented, compared toa case of outputting image data to the plurality of signal processingportions from the imaging element using only one output line.

An imaging apparatus according to a twentieth aspect of the technologyof the present disclosure is an imaging apparatus including an imagingelement and comprises a memory that stores captured image data obtainedby imaging a subject by the imaging element and is incorporated in theimaging element, a processor that is incorporated in the imagingelement, and a plurality of signal processing portions that are disposedoutside the imaging element, in which the processor includes a pluralityof output lines each disposed in correspondence with each of theplurality of signal processing portions and outputs each of a pluralityof pieces of image data into which the captured image data stored in thememory is divided, to a corresponding signal processing portion amongthe plurality of signal processing portions from the plurality of outputlines, and any of the plurality of signal processing portions combinesthe plurality of pieces of image data. Accordingly, high-speed imageprocessing can be implemented, compared to a case of outputting imagedata to the plurality of signal processing portions from the imagingelement using only one output line.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the technology of the disclosure will bedescribed in detail based on the following figures, wherein:

FIG. 1 is a perspective view illustrating an exterior of an imagingapparatus;

FIG. 2 is a rear view illustrating a rear surface side of the imagingapparatus;

FIG. 3 is a block diagram illustrating a configuration of the imagingapparatus;

FIG. 4 is a schematic configuration diagram illustrating a configurationof an imaging element;

FIG. 5 is a block diagram illustrating the configuration of the imagingelement;

FIG. 6 is a block diagram illustrating a flow of data within an imagingapparatus according to a first embodiment;

FIG. 7 is a conceptual diagram illustrating features of a left image anda right image included in a captured image;

FIG. 8 is a block diagram illustrating configurations of a controller, aUI system device, and surroundings of the controller and the UI systemdevice included in the imaging apparatus;

FIG. 9 is a schematic configuration diagram illustrating a configurationof a hybrid finder;

FIG. 10 is a flowchart illustrating a flow of imaging processingaccording to the first embodiment;

FIG. 11 is a flowchart illustrating a flow of first signal processingaccording to the first embodiment;

FIG. 12 is a flowchart illustrating a flow of second signal processingaccording to the first embodiment;

FIG. 13 is a block diagram illustrating a flow of data within an imagingapparatus according to a second embodiment;

FIG. 14A is an image diagram illustrating a relationship between a faceregion image and a background region image;

FIG. 14B is an image diagram illustrating a relationship among the faceregion image, the background region image, and an overlapping region;

FIG. 15 is a flowchart illustrating a flow of imaging processingaccording to the second embodiment;

FIG. 16 is a flowchart illustrating a flow of first signal processingaccording to the second embodiment;

FIG. 17 is a flowchart illustrating a flow of second signal processingaccording to the second embodiment;

FIG. 18 is a block diagram illustrating a flow of data within an imagingapparatus according to a third embodiment;

FIG. 19 is a descriptive diagram used for describing a method ofgenerating vertically thinned image data from captured image data;

FIG. 20 is a conceptual diagram illustrating a relationship among thevertically thinned image data, odd-numbered column image data, andeven-numbered column image data;

FIG. 21 is a conceptual diagram illustrating an aspect of separating thevertically thinned image data into the odd-numbered column image dataand the even-numbered column image data;

FIG. 22 is a flowchart illustrating a flow of imaging processingaccording to the third embodiment;

FIG. 23 is a flowchart illustrating a flow of first signal processingaccording to the third embodiment;

FIG. 24 is a flowchart illustrating a flow of second signal processingaccording to the third embodiment;

FIG. 25 is a block diagram illustrating the flow of data within theimaging apparatus according to the third embodiment;

FIG. 26 is a block diagram illustrating a flow of data within theimaging apparatus in which a still picture image capturing mode is set;

FIG. 27 is a block diagram illustrating a flow of data within theimaging apparatus in which a display image capturing mode is set;

FIG. 28 is a flowchart illustrating a flow of imaging processingaccording to a fourth embodiment;

FIG. 29 is a block diagram illustrating a flow of data within an imagingapparatus according to the fourth embodiment;

FIG. 30 is a descriptive diagram used for describing a method ofgenerating vertically thinned image data from captured image data havingBayer arrangement;

FIG. 31 is a conceptual diagram illustrating a relationship among thevertically thinned image data, first horizontally thinned image data,and second horizontally thinned image data;

FIG. 32 is a block diagram illustrating a form of processing afterseparating the captured image data into high-order bit data andlow-order bit data;

FIG. 33 is a conceptual diagram illustrating a relationship among theimaging element, a plurality of signal processing portions, and thecontroller in a case of using three or more signal processing portions;

FIG. 34 is a conceptual diagram illustrating an example of an aspect inwhich an imaging processing program is installed on a computer withinthe imaging element from a storage medium storing the imaging processingprogram; and

FIG. 35 is a block diagram illustrating an example of a schematicconfiguration of a smart device incorporating the imaging elementaccording to the embodiment.

DETAILED DESCRIPTION

Hereinafter, an example of embodiments of an imaging apparatus accordingto the embodiments of the technology of the present disclosure will bedescribed in accordance with the appended drawings.

First, meanings of terms used in the following description will bedescribed.

In the following description, the abbreviation CPU stands for “CentralProcessing Unit”. In addition, in the following description, theabbreviation RAM stands for “Random Access Memory”. In addition, in thefollowing description, the abbreviation ROM stands for “Read OnlyMemory”. In addition, in the following description, the abbreviationDRAM stands for “Dynamic Random Access Memory”. In addition, in thefollowing description, the abbreviation SRAM stands for “Static RandomAccess Memory”.

In the following description, the abbreviation IC stands for “IntegratedCircuit”. In addition, in the following description, the abbreviationLSI stands for “Large-Scale Integration”. In addition, in the followingdescription, the abbreviation ASIC stands for “Application SpecificIntegrated Circuit”. In addition, in the following description, theabbreviation PLD stands for “Programmable Logic Device”. In addition, inthe following description, the abbreviation FPGA stands for“Field-Programmable Gate Array”.

In the following description, the abbreviation SSD stands for “SolidState Drive”. In addition, in the following description, theabbreviation DVD-ROM stands for “Digital Versatile Disc Read OnlyMemory”. In addition, in the following description, the abbreviation USBstands for “Universal Serial Bus”. In addition, in the followingdescription, the abbreviation HDD stands for “Hard Disk Drive”. Inaddition, in the following description, the abbreviation EEPROM standsfor “Electrically Erasable and Programmable Read Only Memory”.

In the following description, the abbreviation CCD stands for “ChargeCoupled Device”. In addition, in the following description, theabbreviation CMOS stands for “Complementary Metal Oxide Semiconductor”.In addition, in the following description, the abbreviation EL standsfor “Electro-Luminescence”. In addition, in the following description,the abbreviation A/D stands for “Analog/Digital”. In addition, in thefollowing description, the abbreviation FIFO stands for “First in Firstout”. In addition, in the following description, the abbreviation OFstands for “Interface”. In addition, in the following description, theabbreviation EIS stands for “Electronics Image Stabilization”. Inaddition, in the following description, the abbreviation AF stands for“Auto-Focus”. In addition, in the following description, theabbreviation AE stands for “Automatic Exposure”. In addition, in thefollowing description, the abbreviation UI stands for “User Interface”.

First Embodiment

As illustrated in FIG. 1 as an example, an imaging apparatus 10 is aninterchangeable lens camera. The imaging apparatus 10 is a digitalcamera that includes an imaging apparatus main body 12 and aninterchangeable lens 14 interchangeably mounted on the imaging apparatusmain body 12, and that does not include a reflex mirror.

A hybrid finder (registered trademark) 16 is disposed in the imagingapparatus main body 12. For example, the hybrid finder 16 here refers toa finder in which an optical view finder (hereinafter, referred to asthe OVF) and an electronic view finder (hereinafter, referred to as theEVF) are selectively used. The abbreviation OVF stands for “opticalviewfinder”. In addition, the abbreviation EVF stands for “electronicviewfinder”.

A finder switching lever 18 is disposed on a front surface of theimaging apparatus main body 12. An optical image visually recognizableby the OVF and a live view image that is an electronic image visuallyrecognizable by the EVF are switched by rotationally moving the finderswitching lever 18 in a direction of arrow SW. The “live view image”here refers to a motion picture image for displaying obtained by imagingby a photoelectric conversion element 61 (refer to FIG. 3 and FIG. 4 )described later. The live view image is generally referred to as a livepreview image.

A release button 20 and a dial 22 are disposed on an upper surface ofthe imaging apparatus main body 12. The dial 22 is operated in a case ofsetting an operation mode of an imaging system, an operation mode of aplayback system, and the like.

The release button 20 functions as an imaging preparation instructionportion and an imaging instruction portion, and a push operation of twostages of an imaging preparation instruction state and an imaginginstruction state can be detected. For example, the imaging preparationinstruction state refers to a state where a push is performed to anintermediate position (half push position) from a standby position, andthe imaging instruction state refers to a state where a push isperformed to a final push position (full push position) exceeding theintermediate position. Hereinafter, the “state where a push is performedto the half push position from the standby position” will be referred toas a “half push state”, and the “state where a push is performed to thefull push position from the standby position” will be referred to as a“full push state”.

In the imaging apparatus 10, an imaging mode and a playback mode areselectively set as an operation mode in accordance with an instructionof a user. The imaging mode is broadly divided into a display motionpicture capturing mode and a recording imaging mode.

The display motion picture capturing mode is an operation mode in whichthe live view image based on display image data of a plurality ofconsecutive frames obtained by consecutive imaging is displayed on afirst display 32 and/or a second display 86 (refer to FIG. 8 and FIG. 9) described later. The display image data is image data for the liveview image and, for example, is generated by a CPU 46A (refer to FIG. 8) described later based on captured image data 70 (refer to FIG. 3 toFIG. 7 ) indicating an image of a subject. The captured image data 70refers to image data obtained by imaging the subject by an imagingelement 44 (refer to FIG. 3 ) described later. Hereinafter, forconvenience of description, the image indicated by the captured imagedata 70 will be referred to as a “captured image”.

The recording imaging mode is broadly divided into a still picture imagecapturing mode and a motion picture image capturing mode. The stillpicture image capturing mode is an operation mode in which the imagingelement 44 (refer to FIG. 3 ) performs imaging for a still pictureimage. In the still picture image capturing mode, a still picture imageobtained by imaging the subject by the imaging apparatus 10 is recordedon a specific recording device (for example, a secondary storage device(refer to FIG. 8 )). The motion picture image capturing mode is anoperation mode in which the imaging element 44 (refer to FIG. 3 )performs imaging for the motion picture image. In the motion pictureimage capturing mode, a motion picture image obtained by imaging thesubject by the imaging apparatus 10 is stored in the specific recordingdevice.

The recording imaging mode is an operation mode in which the live viewimage is displayed on the first display 32 and/or the second display 86described later, and in which recording image data is recorded on asecondary storage device 80 (refer to FIG. 8 ) described later and/or amemory card or the like. The recording image data is broadly dividedinto still picture image data and motion picture image data and isgenerated based on the captured image data 70 (refer to FIG. 3 to FIG. 7).

In a case where the imaging mode is set, first, the imaging apparatus 10is set to the display motion picture capturing mode. In the displaymotion picture capturing mode, in a case where the push operation isperformed on the release button 20, the imaging apparatus 10 transitionsto the recording imaging mode from the display motion picture capturingmode.

In the imaging mode, a manual focus mode and an auto focus mode areselectively set in accordance with an instruction of the user. In theauto focus mode, an imaging condition is adjusted by setting the releasebutton 20 to the half push state. Then, in a case where the full pushstate is subsequently set, exposure is performed. That is, by settingthe release button 20 to the half push state, an AE function isoperated, and an exposure state is set. Then, an AF function isoperated, and a focusing control is performed. In a case where therelease button 20 is set to the full push state, imaging is performed.

As illustrated in FIG. 2 as an example, a touch panel display 26, aninstruction key 28, and a finder eyepiece portion 30 are disposed on arear surface of the imaging apparatus main body 12.

The touch panel display 26 comprises the first display 32 and a touchpanel 34 (refer to FIG. 8 ). A liquid crystal display or an organic ELdisplay is illustrated as the first display 32.

The first display 32 displays images, text information, and the like.The first display 32 is used for displaying the live view image which isobtained by consecutive imaging in a case where the imaging apparatus 10is in the imaging mode. In addition, the first display 32 is used fordisplaying the still picture image obtained by imaging in a case where astill picture image capturing instruction is provided. Furthermore, thefirst display 32 is used for displaying a playback image and displayinga menu screen and the like in a case where the imaging apparatus 10 isin the playback mode.

The touch panel 34 is a transmissive touch panel and is overlaid on asurface of a display region of the first display 32. The touch panel 34senses a contact by an instruction object such as a finger or a styluspen and outputs a sensing result to a predetermined output destinationsuch as the CPU 46A (refer to FIG. 8 ) described later.

The instruction key 28 receives various instructions such as selectionof one or a plurality of menus, confirmation of a selected content,deletion of the selected content, zooming, and frame advance.

As illustrated in FIG. 3 as an example, the imaging apparatus 10comprises mounts 36 and 38. The mount 36 is disposed in the imagingapparatus main body 12. The mount 38 is disposed in the interchangeablelens 14 at a position corresponding to a position of the mount 36. Theinterchangeable lens 14 is interchangeably mounted on the imagingapparatus main body 12 by joining the mount 38 to the mount 36.

As illustrated in FIG. 3 as an example, the interchangeable lens 14includes an imaging lens 40. The imaging lens 40 comprises an objectivelens 40A, a focus lens 40B, a zoom lens 40C, and a stop 40D. Theobjective lens 40A, the focus lens 40B, the zoom lens 40C, and the stop40D are arranged in an order of the objective lens 40A, the focus lens40B, the zoom lens 40C, and the stop 40D along an optical axis L1 from asubject side to an imaging apparatus main body 12 side. The focus lens40B, the zoom lens 40C, and the stop 40D operate by receiving motivepower from a driving source (not illustrated) such as a motor undercontrol of the CPU 46A (refer to FIG. 8 ) described later. That is, thefocus lens 40B and the zoom lens 40C move along the optical axis L1 inresponse to the provided motive power. In addition, the stop 40D adjustsexposure by operating in response to the provided motive power.

The imaging apparatus main body 12 comprises a mechanical shutter 42 andthe imaging element 44. The mechanical shutter 42 operates by receivingmotive power from a driving source (not illustrated) such as a motorunder control of the CPU 46A (refer to FIG. 8 ) described later. In acase where the interchangeable lens 14 is mounted on the imagingapparatus main body 12 through the mounts 36 and 38, subject lightshowing the subject is transmitted through the imaging lens 40, and animage of the subject light is formed on a light receiving surface 44A ofthe imaging element 44 through the mechanical shutter 42.

The imaging apparatus main body 12 comprises a controller 46, a UIsystem device 48, a first signal processing portion 50, a second signalprocessing portion 52, and DRAMs 54 and 56. The first signal processingportion 50 and the second signal processing portion 52 are an example ofa “plurality of signal processing processors” according to theembodiments of the technology of the present disclosure.

The controller 46 controls the entire imaging apparatus 10. The UIsystem device 48 is a device that presents information to the user orreceives an instruction from the user. The UI system device 48 isconnected to the controller 46. The controller 46 acquires various typesof information from the UI system device 48 and controls the UI systemdevice 48.

The imaging element 44 is connected to the controller 46 through acommunication line 57 and generates the captured image data 70 byimaging the subject under control of the controller 46. As will bedescribed later in detail, the imaging element 44 separates thegenerated captured image data 70 into two pieces of image data. In theexample illustrated in FIG. 3 , first separated image data 70A isillustrated as one piece of image data of the two pieces of image dataobtained by separating the captured image data 70, and second separatedimage data 70B is illustrated as the other piece of image data.

The imaging element 44 is connected to the first signal processingportion 50 through a first output line 53 and is connected to the secondsignal processing portion 52 through a second output line 55. Each ofthe first signal processing portion 50 and the second signal processingportion 52 is an LSI. In the present embodiment, each of the firstsignal processing portion 50 and the second signal processing portion 52is implemented by an ASIC.

However, the technology of the present disclosure is not limitedthereto. For example, a PLD and/or an FPGA may be employed instead ofthe ASIC. In addition, the ASIC, the PLD, and/or the FPGA may beemployed. In addition, a computer including a CPU, a ROM, and a RAM maybe employed. The number of CPUs may be singular or plural. In addition,the first signal processing portion 50 and/or the second signalprocessing portion 52 may be implemented by a combination of a hardwareconfiguration and a software configuration.

The first signal processing portion 50 and the second signal processingportion 52 are connected to each other through a communication line 58.The first signal processing portion 50 is connected to the DRAM 54, andthe second signal processing portion 52 is connected to the DRAM 56. Thefirst signal processing portion 50 is connected to the controller 46through a communication line 60.

The imaging element 44 outputs the first separated image data 70A to thefirst signal processing portion 50 through the first output line 53, andoutputs the second separated image data 70B to the second signalprocessing portion 52 through the second output line 55.

The first signal processing portion 50 performs various types of signalprocessing (for example, a “specific type of signal processing”described later) on the input first separated image data 70A incooperation with the DRAM 54. The second signal processing portion 52performs various types of signal processing (for example, the “specifictype of signal processing” described later) on the input secondseparated image data 70B in cooperation with the DRAM 56. The secondsignal processing portion 52 outputs the second separated image data 70Bsubjected to the various types of signal processing to the first signalprocessing portion 50 through the communication line 58. The firstsignal processing portion 50 combines the first separated image data 70Asubjected to the various types of signal processing with the secondseparated image data 70B input from the second signal processing portion52 and outputs the combined first separated image data 70A and thesecond separated image data 70B to the controller 46 through thecommunication line 60.

The imaging element 44 is an example of a “laminated imaging element”according to the embodiments of the technology of the presentdisclosure. For example, the imaging element 44 is a CMOS image sensor.As illustrated in FIG. 4 as an example, the imaging element 44incorporates the photoelectric conversion element 61, a processingcircuit 62, and a memory 64. The imaging element 44 is an imagingelement in which the photoelectric conversion element 61, the processingcircuit 62, and the memory 64 are formed in one chip. That is, thephotoelectric conversion element 61, the processing circuit 62, and thememory 64 are formed in one package. In the imaging element 44, thephotoelectric conversion element 61 is laminated with the processingcircuit 62 and the memory 64. Specifically, the photoelectric conversionelement 61 and the processing circuit 62 are electrically connected toeach other by a bump (not illustrated) of copper or the like havingconductivity. The processing circuit 62 and the memory 64 are alsoelectrically connected to each other by a bump (not illustrated) ofcopper or the like having conductivity. The memory 64 is an example of astorage portion according to the embodiments of the technology of thepresent disclosure.

The processing circuit 62 is, for example, an LSI, and the memory 64 is,for example, a DRAM. However, the technology of the present disclosureis not limited thereto, and an SRAM may be employed as the memory 64instead of the DRAM.

The processing circuit 62 is implemented by an ASIC and controls theentire imaging element 44 in accordance with an instruction of thecontroller 46. While an example of implementing the processing circuit62 by the ASIC is illustrated here, the technology of the presentdisclosure is not limited thereto. For example, a PLD and/or an FPGA maybe employed instead of the ASIC. In addition, the ASIC, the PLD, and/orthe FPGA may be employed. In addition, a computer including a CPU, aROM, and a RAM may be employed. The number of CPUs may be singular orplural. In addition, the processing circuit 62 may be implemented by acombination of a hardware configuration and a software configuration.

The photoelectric conversion element 61 includes a plurality ofphotodiodes arranged in a matrix form. Photodiodes of “4896×3265” pixelsare illustrated as an example of the plurality of photodiodes.

The photoelectric conversion element 61 comprises color filters, and thecolor filters include a G filter corresponding to green (G) that mostcontributes to obtaining a brightness signal, an R filter correspondingto red (R), and a B filter corresponding to blue (B). In the presentembodiment, the G filter, the R filter, and the B filter are arrangedwith a predetermined periodicity in each of a row direction (horizontaldirection) and a column direction (vertical direction) for the pluralityof photodiodes of the photoelectric conversion element 61. Thus, theimaging apparatus 10 can perform processing in accordance with arepeating pattern in a case of performing demosaicing and the like on R,G, and B signals. The demosaicing refers to processing of calculatingthe entire color information for each pixel from a mosaic imagecorresponding to color filter arrangement of a single plate colorimaging element. For example, in a case of an imaging element consistingof color filters of three colors of R, G, and B, the demosaicing meansprocessing of calculating color information about all of R, G, and B foreach pixel from a mosaic image consisting of R, G, and B.

While the CMOS image sensor is illustrated here as the imaging element44, the technology of the present disclosure is not limited thereto. Forexample, the technology of the present disclosure is also established ina case where the imaging element 44 is a CCD image sensor.

The imaging element 44 has a so-called electronic shutter function andcontrols an electric charge accumulation time period of each photodiodein the photoelectric conversion element 61 by performing the electronicshutter function under control of the controller 46. The electric chargeaccumulation time period refers to a so-called shutter speed.

In the imaging apparatus 10, the imaging for the still picture image andthe imaging for the motion picture image are performed using a rollingshutter method. The imaging for the still picture image is implementedby performing the electronic shutter function and operating themechanical shutter 42 (refer to FIG. 3 ). The imaging for the motionpicture is implemented by performing the electronic shutter functionwithout operating the mechanical shutter 42. While the rolling shuttermethod is illustrated here, the technology of the present disclosure isnot limited thereto. A global shutter method may be applied instead ofthe rolling shutter method.

The processing circuit 62 reads out the captured image data 70 obtainedby imaging the subject by the photoelectric conversion element 61. Thecaptured image data 70 is signal electric charges accumulated in thephotoelectric conversion element 61. The processing circuit 62 performsA/D conversion on the captured image data 70 read out from thephotoelectric conversion element 61. The processing circuit 62 storesthe captured image data 70 obtained by performing the A/D conversion onthe captured image data 70 in the memory 64.

The processing circuit 62 acquires the captured image data 70 from thememory 64 and performs various types of processing on the acquiredcaptured image data 70. The “various types of processing” here includesprocessing of separating the captured image data 70 into the firstseparated image data 70A and the second separated image data 70B asillustrated in FIG. 4 . The processing circuit 62 outputs the firstseparated image data 70A to the first signal processing portion 50through the first output line 53, and outputs the second separated imagedata 70B to the second signal processing portion 52 through the secondoutput line 55.

As illustrated in FIG. 5 as an example, the processing circuit 62includes a photoelectric conversion element control circuit 62A, adigital processing circuit 62B, an image processing circuit 62C, and anoutput circuit 62D. The output circuit 62D is an example of an “outputportion” according to the embodiments of the technology of the presentdisclosure.

The photoelectric conversion element control circuit 62A is connected tothe photoelectric conversion element 61 and the digital processingcircuit 62B. The memory 64 is connected to the digital processingcircuit 62B and the image processing circuit 62C. The image processingcircuit 62C is connected to the output circuit 62D and the memory 64.

The output circuit 62D includes the first output line 53 and the secondoutput line 55. The first output line 53 corresponds to the first signalprocessing portion 50 and connects the output circuit 62D to the firstsignal processing portion 50. The second output line 55 corresponds tothe second signal processing portion 52 and connects the output circuit62D to the second signal processing portion 52.

The photoelectric conversion element control circuit 62A controls thephotoelectric conversion element 61 and reads out the analog capturedimage data 70 from the photoelectric conversion element 61 under controlof the controller 46. The digital processing circuit 62B digitizes theanalog captured image data 70 by performing signal processing ofcorrelative double sampling processing and then, the A/D conversion onthe analog captured image data 70 read out by the photoelectricconversion element control circuit 62A. The digital processing circuit62B stores the digitized captured image data 70 in the memory 64.

The memory 64 is a memory that can store the captured image data 70 of aplurality of frames. The memory 64 has a storage region (notillustrated) in units of pixels. The captured image data 70 is stored ina corresponding storage region of the memory 64 in units of pixels bythe digital processing circuit 62B.

The image processing circuit 62C acquires the captured image data 70from the memory 64 and processes the acquired captured image data 70.

The image processing circuit 62C performs the various types ofprocessing on the captured image data 70. The image processing circuit62C separates the captured image data 70 into the first separated imagedata 70A and the second separated image data 70B and outputs the firstseparated image data 70A and the second separated image data 70B (referto FIG. 3 and FIG. 4 ) obtained by separation to the output circuit 62D.

The output circuit 62D outputs the first separated image data 70A inputfrom the image processing circuit 62C to the first signal processingportion 50 through the first output line 53. In addition, the outputcircuit 62D outputs the second separated image data 70B input from theimage processing circuit 62C to the second signal processing portion 52through the second output line 55.

An output frame rate in the output circuit 62D is a frame rate that isthe same as a frame rate used in a device in a rear stage of the imagingelement 44. The output frame rate is a frame rate required foroutputting the first separated image data 70A and the second separatedimage data 70B by the output circuit 62D and is, for example, 60 framesper second (fps). Meanwhile, an imaging frame rate is a frame raterequired for imaging performed by cooperation among the photoelectricconversion element 61, the photoelectric conversion element controlcircuit 62A, the digital processing circuit 62B, and the memory 64 andis, for example, 120 fps. The “imaging” here refers to processing from astart of exposure of one frame in the photoelectric conversion element61 to storage of the captured image data 70 of one frame in the memory64.

Here, specific processing contents in the image processing circuit 62C,the output circuit 62D, the first signal processing portion 50, and thesecond signal processing portion 52 will be described.

As illustrated in FIG. 6 as an example, the image processing circuit 62Cacquires the captured image data 70 from the memory 64 and separates theacquired captured image data 70 into left image data 70A1 and rightimage data 70B1. The image processing circuit 62C outputs the left imagedata 70A1 and the right image data 70B1, which are obtained byseparating the captured image data 70, to the output circuit 62D.

The left image data 70A1 is an example of the first separated image data70A (refer to FIG. 3 to FIG. 5 ), and the right image data 70B1 is anexample of the second separated image data 70B (refer to FIG. 3 to FIG.5 ).

The left image data 70A1 is image data indicating a left image 70A1 a(refer to FIG. 7 ), and the right image data 70B1 is image dataindicating a right image 70B1 a (refer to FIG. 7 ). As illustrated inFIG. 7 as an example, the left image 70A1 a and the right image 70B1 aare a pair of images adjacent left and right to each other. The leftimage 70A1 a and the right image 70B1 a have an overlapping region 71.The overlapping region 71 is a region that overlaps between the leftimage 70A1 a and the right image 70B1 a in a left-right direction RL.For example, the number of pixels of the overlapping region 71 in theleft-right direction RL is a few tens of pixels to a few hundred pixels.

In the example illustrated in FIG. 6 , an example of a form of acquiringthe captured image data 70 from the memory 64 and separating theacquired captured image data 70 by the image processing circuit 62C isillustrated. However, the technology of the present disclosure is notlimited thereto. In this case, for example, first, the image processingcircuit 62C selects the left image data 70A1 and the right image data70B1 from the captured image data 70 in accordance with a predeterminedaddress in the memory 64. The image processing circuit 62C directlyacquires the selected left image data 70A1 and the right image data 70B1from the memory 64. The predetermined address includes an address foracquiring the left image data 70A1 and an address for acquiring theright image data 70B1. The address for acquiring the left image data70A1 and the address for acquiring the right image data 70B1 aredetermined such that image data indicating the overlapping region 71 isalso included in each of the left image data 70A1 and the right imagedata 70B1.

The first signal processing portion 50 includes a buffer 50A, a signalprocessing circuit 50B, and a reception circuit 50C. The DRAM 54 and thecontroller 46 are connected to the signal processing circuit 50B. Thesecond signal processing portion 52 includes a buffer 52A, a signalprocessing circuit 52B, and a transmission circuit 52C. The DRAM 56 isconnected to the signal processing circuit 52B.

The output circuit 62D outputs the left image data 70A1 to the buffer50A through the first output line 53. The buffer 50A holds the leftimage data 70A1 which is input and outputs the left image data 70A1 tothe signal processing circuit 50B using a FIFO method. The signalprocessing circuit 50B stores the left image data 70A1 input from thebuffer 50A in the DRAM 54. The signal processing circuit 50B performssignal processing (hereinafter, referred to as the “specific type ofsignal processing”) such as tone correction, white balance adjustment,sharpness adjustment, gamma correction, and gradation correction on theleft image data 70A1 stored in the DRAM 54.

The output circuit 62D outputs the right image data 70B1 to the buffer52A through the second output line 55. The buffer 52A holds the rightimage data 70B1 which is input and outputs the right image data 70B1 tothe signal processing circuit 52B using the FIFO method. The signalprocessing circuit 52B stores the right image data 70B1 input from thebuffer 52A in the DRAM 56. The signal processing circuit 52B performsthe specific type of signal processing on the right image data 70B1stored in the DRAM 56.

The transmission circuit 52C transmits the right image data 70B1subjected to the specific type of signal processing by the signalprocessing circuit 52B to the first signal processing portion 50. In thefirst signal processing portion 50, the reception circuit 50C receivesthe right image data 70B1 transmitted from the transmission circuit 52C.

The signal processing circuit 50B generates combined image data 72 bycombining the right image data 70B1 received by the reception circuit50C with the left image data 70A1 subjected to the signal processing.The signal processing circuit 50B outputs the combined image data 72obtained by combining to the controller 46 through the communicationline 60.

The combined image data 72 is generated by joining the left image data70A1 to the right image data 70B1. Here, an arithmetic mean of imagedata indicating the overlapping region 71 (refer to FIG. 7 ) in the leftimage data 70A1 and image data indicating the overlapping region 71(refer to FIG. 7 ) in the right image data 70B1 is calculated.Accordingly, noticeability of a boundary region between two images issuppressed, compared to a case of joining two images obtained by simplydividing a captured image into two parts in the left-right direction RL(refer to FIG. 7 ). While the arithmetic mean is illustrated here, thetechnology of the present disclosure is not limited thereto.Substitution may be employed instead of the arithmetic mean. The“substitution” here refers to replacing one of the image data indicatingthe overlapping region 71 in the left image data 70A1 and the image dataindicating the overlapping region 71 in the right image data 70B1 withthe other.

As illustrated in FIG. 8 as an example, the controller 46 comprises aCPU 46A, a ROM 46B, a RAM 46C, a connection I/F 46D, and an input I/F46E. The CPU 46A, the ROM 46B, the RAM 46C, the connection I/F 46D, andthe input I/F 46E are connected to each other through a busline 88.

The ROM 46B stores various programs. The CPU 46A reads out the variousprograms from the ROM 46B and loads the read various programs into theRAM 46C. The CPU 46A controls the entire imaging apparatus 10 inaccordance with the various programs loaded in the RAM 46C.

The connection I/F 46D is an FPGA and is connected to the imagingelement 44 through the communication line 57. The CPU 46A controls theimaging element 44 through the connection I/F 46D.

The input I/F 46E is a device including an FPGA and is connected to thefirst signal processing portion 50 through the communication line 60.The combined image data 72 (refer to FIG. 6 ) is input into the inputI/F 46E from the first signal processing portion 50 through thecommunication line 60. The input I/F 46E transfers the combined imagedata 72 input from the first signal processing portion 50 to the CPU46A.

The secondary storage device 80 and an external I/F 82 are connected tothe busline 88. The secondary storage device 80 is a non-volatile memorysuch as an SSD, an HDD, or an EEPROM. The CPU 46A reads out and writesvarious types of information in the secondary storage device 80.

The external I/F 82 is a device including an FPGA. An external apparatus(not illustrated) such as a USB memory and a memory card is connected tothe external I/F 82. The external I/F 82 exchanges various types ofinformation between the CPU 46A and the external apparatus.

The UI system device 48 comprises the hybrid finder 16, the touch paneldisplay 26, and a reception portion 84. The first display 32 and thetouch panel 34 are connected to the busline 88. Accordingly, the CPU 46Adisplays various types of information on the first display 32 andoperates in accordance with various instructions received by the touchpanel 34.

The reception portion 84 comprises the touch panel 34 and a hard keyportion 25. The hard key portion 25 includes a plurality of hard keysand includes a release button 20, a dial 22, and an instruction key 28.The hard key portion 25 is connected to the busline 88, and the CPU 46Aoperates in accordance with various instructions received by the hardkey portion 25.

The hybrid finder 16 comprises the second display 86. The CPU 46Adisplays various types of information on the second display 86.

As illustrated in FIG. 9 as an example, the hybrid finder 16 includes anOVF 90 and an EVF 92. The OVF 90 is a reverse Galilean finder andincludes an eyepiece lens 94, a prism 96, and an objective lens 98. TheEVF 92 includes the second display 86, the prism 96, and the eyepiecelens 94.

A liquid crystal shutter 100 is arranged closer to the subject side thanthe objective lens 98 along an optical axis L2 of the objective lens 98.The liquid crystal shutter 100 blocks light such that the optical imageis not incident on the objective lens 98 in a case of using the EVF 92.

The prism 96 reflects and guides the electronic image or various typesof information displayed on the second display 86 to the eyepiece lens94 and combines the optical image with the electronic image and/or thevarious types of information displayed on the second display 86. A liveview image 102 based on the combined image data 72 is illustrated as theelectronic image displayed on the second display 86.

In a case of an OVF mode, the CPU 46A enables the optical image to bevisually recognized from the eyepiece lens 94 by controlling the liquidcrystal shutter 100 to a non-light blocking state. In addition, in acase of an EVF mode, the CPU 46A enables only the electronic imagedisplayed on the second display 86 to be visually recognized from theeyepiece lens 94 by controlling the liquid crystal shutter 100 to alight blocking state.

Hereinafter, for convenience of description, the first display 32 (referto FIG. 2 and FIG. 8 ) and the second display 86 will be referred to asthe “display” without a reference sign unless otherwise necessary todistinguish therebetween. The display is an example of a “displayportion” according to the embodiments of the technology of the presentdisclosure. In addition, the CPU 46A is an example of a “control portion(control processor)” according to the embodiments of the technology ofthe present disclosure.

Next, an action of the imaging apparatus 10 will be described.

First, a flow of imaging processing executed by the processing circuit62 of the imaging element 44 will be described with reference to FIG. 10.

In the imaging processing illustrated in FIG. 10 , first, in step ST10,the photoelectric conversion element control circuit 62A determineswhether or not a timing (hereinafter, referred to as an “exposure starttiming”) at which the photoelectric conversion element 61 starts theexposure is reached. The exposure start timing is a timing that isperiodically defined by the imaging frame rate. In step ST10, in a casewhere the exposure start timing is not reached, a negative determinationis made, and the imaging processing transitions to step ST22. In stepST10, in a case where the exposure start timing is reached, a positivedetermination is made, and the imaging processing transitions to stepST12.

In step ST12, the photoelectric conversion element control circuit 62Acauses the photoelectric conversion element 61 to perform the exposureof one frame.

In subsequent step ST14, the photoelectric conversion element controlcircuit 62A reads out the captured image data 70 of one frame from thephotoelectric conversion element 61.

In subsequent step ST16, the digital processing circuit 62B digitizesthe analog captured image data 70 by performing signal processing of thecorrelative double sampling processing and then, the A/D conversion onthe captured image data 70 read out in step ST14. The digital processingcircuit 62B stores the digitized captured image data 70 in the memory64.

In subsequent step ST18, the image processing circuit 62C acquires thecaptured image data 70 from the memory 64 and separates the acquiredcaptured image data 70 into the left image data 70A1 (refer to FIG. 6 )and the right image data 70B1 (refer to FIG. 6 ). The image processingcircuit 62C outputs the left image data 70A1 and the right image data70B1 to the output circuit 62D.

In subsequent step ST20, the output circuit 62D outputs the left imagedata 70A1 to the first signal processing portion 50 through the firstoutput line 53 (refer to FIG. 3 to FIG. 6 and FIG. 8 ). In addition, theoutput circuit 62D outputs the right image data 70B1 to the secondsignal processing portion 52 through the second output line 55 (refer toFIG. 3 to FIG. 6 and FIG. 8 ).

In subsequent step ST22, the processing circuit 62 determines whether ornot a condition (hereinafter, referred to as an “imaging processingfinish condition”) under which the imaging processing is finished issatisfied. For example, a condition that an instruction to finish theimaging processing is received by the reception portion 84 isillustrated as the imaging processing finish condition. In step ST22, ina case where the imaging processing finish condition is not satisfied, anegative determination is made, and the imaging processing transitionsto step ST10. In step ST22, in a case where the imaging processingfinish condition is satisfied, a positive determination is made, and theimaging processing is finished.

Next, a flow of first signal processing executed by the first signalprocessing portion 50 will be described with reference to FIG. 11 .

In the first signal processing illustrated in FIG. 11 , in step ST30,the first signal processing portion 50 determines whether or not theleft image data 70A1 (refer to FIG. 6 ) is input from the processingcircuit 62. In step ST30, in a case where the left image data 70A1 isnot input from the processing circuit 62, a negative determination ismade, and the first signal processing transitions to step ST42. In stepST30, in a case where the left image data 70A1 is input from theprocessing circuit 62, a positive determination is made, and the firstsignal processing transitions to step ST32.

In step ST32, the first signal processing portion 50 performs thespecific type of signal processing on the left image data 70A1.

In subsequent step ST34, the first signal processing portion 50determines whether or not the right image data 70B1 (refer to FIG. 6 )transmitted by executing processing of step ST54 of second signalprocessing illustrated in FIG. 12 is received. In step ST34, in a casewhere the right image data 70B1 is not received, a negativedetermination is made, and the first signal processing transitions tostep ST40. In step ST34, in a case where the right image data 70B1 isreceived, a positive determination is made, and the first signalprocessing transitions to step ST36.

In step ST40, the first signal processing portion 50 determines whetheror not a condition (hereinafter, referred to as a “first signalprocessing finish condition”) under which the first signal processing isfinished is satisfied. For example, a condition that an instruction tofinish the first signal processing is received by the reception portion84 is illustrated as the first signal processing finish condition. Instep ST40, in a case where the first signal processing finish conditionis not satisfied, a negative determination is made, and the first signalprocessing transitions to step ST34. In step ST40, in a case where thefirst signal processing finish condition is satisfied, a positivedetermination is made, and the first signal processing is finished.

In step ST36, the first signal processing portion 50 generates thecombined image data 72 (refer to FIG. 6 ) by performing combiningprocessing of combining the left image data 70A1 obtained by executingprocessing of step ST32 with the right image data 70B1 received in stepST34.

In subsequent step ST38, the first signal processing portion 50 outputsthe combined image data 72 obtained by executing processing of step ST36to the controller 46 (refer to FIG. 6) through the communication line 60(refer to FIG. 6 and FIG. 8 ).

In subsequent step ST42, the first signal processing portion 50determines whether or not the first signal processing finish conditionis satisfied. In step ST42, in a case where the first signal processingfinish condition is not satisfied, a negative determination is made, andthe first signal processing transitions to step ST30. In step ST42, in acase where the first signal processing finish condition is satisfied, apositive determination is made, and the first signal processing isfinished.

Next, a flow of second signal processing executed by the second signalprocessing portion 52 will be described with reference to FIG. 12 .

In the second signal processing illustrated in FIG. 12 , in step ST50,the second signal processing portion 52 determines whether or not theright image data 70B1 (refer to FIG. 6 ) is input from the processingcircuit 62. In step ST50, in a case where the right image data 70B1 isnot input from the processing circuit 62, a negative determination ismade, and the second signal processing transitions to step ST56. In stepST50, in a case where the right image data 70B1 is input from theprocessing circuit 62, a positive determination is made, and the secondsignal processing transitions to step ST52.

In step ST52, the second signal processing portion 52 performs thespecific type of signal processing on the right image data 70B1.

In subsequent step ST54, the second signal processing portion 52transmits the right image data 70B1 obtained by executing processing ofstep ST52 to the first signal processing portion 50 through thecommunication line 58 (refer to FIG. 3 to FIG. 6 and FIG. 8 ).

In subsequent step ST56, the second signal processing portion 52determines whether or not a condition (hereinafter, referred to as a“second signal processing finish condition”) under which the secondsignal processing is finished is satisfied. For example, a conditionthat an instruction to finish the second signal processing is receivedby the reception portion 84 is illustrated as the second signalprocessing finish condition. In step ST56, in a case where the secondsignal processing finish condition is not satisfied, a negativedetermination is made, and the second signal processing transitions tostep ST50. In step ST56, in a case where the second signal processingfinish condition is satisfied, a positive determination is made, and thesecond signal processing is finished.

As described above, the imaging apparatus 10 comprises the imagingelement 44, the first signal processing portion 50, and the secondsignal processing portion 52. In addition, the imaging element 44comprises the output circuit 62D. The left image data 70A1 is output tothe first signal processing portion 50 through the first output line 53by the output circuit 62D, and the right image data 70B1 is output tothe second signal processing portion 52 through the second output line55 by the output circuit 62D. Each of the left image data 70A1 and theright image data 70B1 subjected to the specific type of signalprocessing is combined by the first signal processing portion 50, andthe combined image data 72 obtained by combining is output to thecontroller 46.

Thus, a traffic between the imaging element 44 and each of the firstsignal processing portion 50 and the second signal processing portion 52is decreased, compared to a case of outputting the captured image data70 to the first signal processing portion 50 and the second signalprocessing portion 52 using only one output line.

In addition, a data amount of image data of a target on which each ofthe first signal processing portion 50 and the second signal processingportion 52 performs the specific type of signal processing is smallerthan the captured image data 70. Thus, a load exerted on each of thefirst signal processing portion 50 and the second signal processingportion 52 in a case of executing the specific type of signal processingis reduced, compared to a case of performing the specific type of signalprocessing on the entire captured image data 70 by only the first signalprocessing portion 50 or the second signal processing portion 52.

Accordingly, the imaging apparatus 10 can implement high-speed imageprocessing, compared to a case of outputting image data to a pluralityof signal processing portions from the imaging element 44 using only oneoutput line.

In addition, as illustrated in FIG. 7 , each of the left image data 70A1and the right image data 70B1 is image data indicating an image havingthe overlapping region 71 between the left image 70A1 a and the rightimage 70B1 a. In a case where two images obtained by simply dividing thecaptured image into two parts in the left-right direction RL (refer toFIG. 7 ) are combined, there is a concern that the boundary regionbetween two images is noticeable. However, as illustrated in FIG. 7 ,each of the left image 70A1 a and the right image 70B1 a has theoverlapping region 71. Thus, an occurrence of an event such that theboundary region between two images is noticeable can be suppressed,compared to a case of combining two images obtained by simply dividingthe captured image into two parts in the left-right direction RL.

In addition, the imaging element 44 is an imaging element in which thephotoelectric conversion element 61, the processing circuit 62, and thememory 64 are formed in one chip. Accordingly, portability of theimaging element 44 is increased, compared to an imaging element in whichthe photoelectric conversion element 61, the processing circuit 62, andthe memory 64 are not formed in one chip. In addition, a degree ofdesign freedom can be increased, compared to a case of the imagingelement in which the photoelectric conversion element 61, the processingcircuit 62, and the memory 64 are not formed in one chip. Furthermore,it is possible to contribute to size reduction of the imaging apparatusmain body 12, compared to a case of the imaging element in which thephotoelectric conversion element 61, the processing circuit 62, and thememory 64 are not formed in one chip.

In addition, as illustrated in FIG. 4 , the laminated imaging element inwhich the photoelectric conversion element 61 is laminated with thememory 64 is employed as the imaging element 44. Accordingly, a loadexerted on processing between the photoelectric conversion element 61and the memory 64 can be reduced, compared to a case of not laminatingthe photoelectric conversion element 61 and the memory 64. In addition,the degree of design freedom can be increased, compared to a case of notlaminating the photoelectric conversion element 61 and the memory 64.Furthermore, it is possible to contribute to size reduction of theimaging apparatus main body 12, compared to a case of not laminating thephotoelectric conversion element 61 and the memory 64.

In addition, as illustrated in FIG. 9 , an image indicated by thecombined image data 72 is displayed as the live view image 102 on thesecond display 86 under control of the CPU 46A. Accordingly, the usercan visually recognize the image indicated by the combined image data72. While the live view image 102 is displayed on the second display 86in the example illustrated in FIG. 9 , the technology of the presentdisclosure is not limited thereto. For example, the live view image 102may be displayed on the first display 32, or the live view image 102 maybe displayed on both of the first display 32 and the second display 86.

While an imaging element in which the photoelectric conversion element61, the processing circuit 62, and the memory 64 are formed in one chipis illustrated as the imaging element 44 in the first embodiment, thetechnology of the present disclosure is not limited thereto. Forexample, at least the photoelectric conversion element 61 and the memory64 among the photoelectric conversion element 61, the processing circuit62, and the memory 64 may be formed in one chip.

While an example of a form of separating the captured image data 70 intothe left image data 70A1 and the right image data 70B1 is illustrativelydescribed in the first embodiment, the technology of the presentdisclosure is not limited thereto. For example, the captured image data70 may be separated into upper image data indicating an image of anupper region of the captured image and lower image data indicating animage of a lower region of the captured image. In addition, the capturedimage data 70 may be separated into upper left image data indicating animage of an upper left region of the captured image and lower rightimage data indicating an image of a lower right region of the capturedimage. In addition, the captured image data 70 may be separated intoupper right image data indicating an image of an upper right region ofthe captured image and lower left image data indicating an image of alower left region of the captured image. Furthermore, the captured imagedata 70 may be separated into center image data indicating an image of acenter region of the captured image and a peripheral region of thecaptured image, that is, peripheral image data indicating an image of aregion other than the center region.

Even in a case of separating the captured image data 70 in such amanner, it is preferable that each of a pair of pieces of image dataobtained by separating the captured image data 70 is image dataincluding overlapping image data indicating an image overlapping in aseparation direction between two images into which the captured image isseparated.

Second Embodiment

While a case of performing the same signal processing on each of thefirst separated image data 70A and the second separated image data 70Bis described in the first embodiment, a case of performing signalprocessing on the first separated image data 70A and the secondseparated image data 70B in a discriminatory manner will be described ina second embodiment. In the second embodiment, the same constituents asthe first embodiment will be designated by the same reference signs andwill not be described. Hereinafter, parts different from the firstembodiment will be described.

As illustrated in FIG. 1 , an imaging apparatus 200 according to thesecond embodiment is different from the imaging apparatus 10 describedin the first embodiment in that an imaging apparatus main body 212 isincluded instead of the imaging apparatus main body 12.

The imaging apparatus main body 212 is different from the imagingapparatus main body 12 in that an imaging element 244 (refer to FIG. 13) is included instead of the imaging element 44. In addition, theimaging apparatus main body 212 is different from the imaging apparatusmain body 12 in that a first signal processing portion 250 (refer toFIG. 13 ) is included instead of the first signal processing portion 50,and that a second signal processing portion 252 (refer to FIG. 13 ) isincluded instead of the second signal processing portion 52.

As illustrated in FIG. 13 , the imaging element 244 is different fromthe imaging element 44 in that a processing circuit 262 is includedinstead of the processing circuit 62. The processing circuit 262 isdifferent from the processing circuit 62 in that an image processingcircuit 262C is included instead of the image processing circuit 62C,and that an output circuit 262D is included instead of the outputcircuit 62D.

The image processing circuit 262C is different from the image processingcircuit 62C in that a face detection circuit 262C1 is included. The facedetection circuit 262C1 is an example of a “detection portion (detectionprocessor)” according to the embodiments of the technology of thepresent disclosure. The face detection circuit 262C1 is a circuit havinga well-known face detection function. The face detection circuit 262C1is implemented by a hardware configuration based on an ASIC or the like.The face detection circuit 262C1 is not limited to the hardwareconfiguration, and may be implemented by a software configuration or maybe implemented by the software configuration and the hardwareconfiguration. In the second embodiment, the face detection circuit262C1 acquires the captured image data 70 from the memory 64. The facedetection circuit 262C1 specifies a face image 69 (refer to FIG. 14A)indicating a face of a person in the captured image indicated by theacquired captured image data 70. That is, the face detection circuit262C1 detects face image data indicating the face image 69 from thecaptured image data 70.

The face detection circuit 262C1 separates the captured image data 70into face region image data 70B2 and background region image data 70A2by extracting the face region image data 70B2 indicating a face regionimage 70B2 a (refer to FIG. 14A) within a predetermined range includingthe specified face image 69 from the captured image data 70.

The background region image data 70A2 refers to an image of a backgroundregion of the captured image, that is, image data indicating an imageother than the face region image 70B2 a. In addition, for example, thepredetermined range refers to a range that is determined in accordancewith a position of the face image 69 (refer to FIG. 14A) in the capturedimage and a degree of difference between a size of the face image 69 anda size of an image other than the face image 69 of the captured image.In the example illustrated in FIG. 14A, the face image 69 is positionedin the center region of the captured image. In this case, the capturedimage is separated into three regions including the center region, theupper region, and the lower region. The image of the center region isspecified as the face region image 70B2 a, and the image of the upperregion and the image of the lower region, that is, the image other thanthe center region of the captured image, are specified as the backgroundregion image 70A2 a.

In the example illustrated in FIG. 14A, an image showing faces of twopersons is illustrated as the face image 69. The face image 69 is animage for specifying a region having the highest density of faces of aplurality of persons in the captured image. The technology of thepresent disclosure is not limited thereto. For example, instead of theface image 69, a face image showing a face of one person may be applied,a face image showing a face of a specific facial expression (forexample, a smiling face) may be applied, or a face image showing a faceof a specific person may be applied.

As illustrated in FIG. 13 , the background region image data 70A2 andthe face region image data 70B2 are output to the output circuit 262D bythe image processing circuit 262C. The background region image data 70A2is an example of the first separated image data 70A, and the face regionimage data 70B2 is an example of the second separated image data 70B. Inaddition, the face region image 70B2 a is an example of a “designatedimage” according to the embodiments of the technology of the presentdisclosure, and the background region image 70A2 a is an example of an“image different from the designated image” according to the embodimentsof the technology of the present disclosure.

The output circuit 262D is different from the output circuit 62D in thatthe background region image data 70A2 is output instead of the leftimage data 70A1, and that the face region image data 70B2 is outputinstead of the right image data 70B1.

Specifically, the output circuit 262D outputs the background regionimage data 70A2 to the first signal processing portion 250 through thefirst output line 53. In addition, the output circuit 262D outputs theface region image data 70B2 to the second signal processing portion 252through the second output line 55.

The first signal processing portion 250 is different from the firstsignal processing portion 50 in that a signal processing circuit 250B isincluded instead of the signal processing circuit 50B.

The background region image data 70A2 output from the output circuit262D is temporarily held in the buffer 50A and is output to the signalprocessing circuit 250B from the buffer 50A. The signal processingcircuit 250B performs the specific type of signal processing on thebackground region image data 70A2 input from the buffer 50A.

The second signal processing portion 252 is different from the secondsignal processing portion 52 in that a second signal processing circuit252B is included instead of the signal processing circuit 52B. Thesignal processing circuit 252B is different from the signal processingcircuit 52B in that a face authentication processing circuit 252B1 isincluded.

The face region image data 70B2 output from the output circuit 262D istemporarily held in the buffer 52A and is output to the signalprocessing circuit 252B from the buffer 52A. The signal processingcircuit 252B performs the specific type of signal processing on the faceregion image data 70B2 input from the buffer 52A. The faceauthentication processing circuit 252B1 has a well-known faceauthentication function and executes face authentication on the faceregion image data 70B2 subjected to the specific type of signalprocessing. By executing the face authentication, for example, adetermination as to whether or not the face shown by the face image 69(refer to FIG. 14A) corresponds to a specific display (for example, asmiling face), and/or a determination as to whether or not the faceshown by the face image 69 is a face of a specific person is performed.

The face authentication processing circuit 252B1 outputs the face regionimage data 70B2 on which the face authentication is executed, to thetransmission circuit 52C. Face authentication result information thatindicates a result of the face authentication is assigned to the faceregion image data 70B2 and is used for processing in a rear stagecircuit. For example, the “rear stage circuit” here refers to the firstsignal processing portion 250 and/or the controller 46.

The transmission circuit 52C transmits the face region image data 70B2input from the face authentication processing circuit 252B1 to the firstsignal processing portion 250.

In the first signal processing portion 250, the reception circuit 50Creceives the face region image data 70B2 transmitted from thetransmission circuit 52C. The signal processing circuit 250B generatescombined image data 272 by combining the face region image data 70B2received by the reception circuit 50C with the background region imagedata 70A2 subjected to the specific type of signal processing by thesignal processing circuit 250B. The signal processing circuit 250Boutputs the combined image data 272 to the controller 46 through thecommunication line 60.

As illustrated in FIG. 14B as an example, the background region image70A2 a and the face region image 70B2 a may include an overlappingregion 271. The overlapping region 271 is a region that overlaps betweenthe background region image 70A2 a and the face region image 70B2 a inan up-down direction UD. In the same manner as the overlapping region 71described in the first embodiment, for example, the number of pixels ofthe overlapping region 271 in the up-down direction UD may be a few tensof pixels to a few hundred pixels. In a case of combining the backgroundregion image data 70A2 with the face region image data 70B2 by the firstsignal processing portion 250, an arithmetic mean of each image dataindicating the overlapping region 271 of each of the background regionimage data 70A2 and the face region image data 70B2 is calculated in thesame manner as the first embodiment.

Next, an action of the imaging apparatus 200 will be described.

First, a flow of imaging processing executed by the processing circuit262 of the imaging element 244 will be described with reference to FIG.15 . The imaging processing illustrated in FIG. 15 is different from theimaging processing illustrated in FIG. 10 in that processing of stepST60 is included instead of processing of step ST18, and that step ST62is included instead of processing of step ST20. Thus, in a flowchart ofthe imaging processing illustrated in FIG. 15 , the same steps as theimaging processing illustrated in FIG. 10 are designated by the samestep numbers. Hereinafter, only parts of the imaging processingillustrated in FIG. 15 different from the imaging processing illustratedin FIG. 10 will be described.

In the imaging processing illustrated in FIG. 15 , in step ST60, theface detection circuit 262C1 acquires the captured image data 70 fromthe memory 64 and separates the acquired captured image data 70 into thebackground region image data 70A2 and the face region image data 70B2.The image processing circuit 262C outputs the background region imagedata 70A2 and the face region image data 70B2 to the output circuit262D.

In subsequent step ST62, the output circuit 262D outputs the backgroundregion image data 70A2 to the first signal processing portion 250through the first output line 53. In addition, the output circuit 262Doutputs the face region image data 70B2 to the second signal processingportion 252 through the second output line 55.

Next, a flow of first signal processing executed by the first signalprocessing portion 250 will be described with reference to FIG. 16 .

In the first signal processing illustrated in FIG. 16 , in step ST70,the first signal processing portion 250 determines whether or not thebackground region image data 70A2 (refer to FIG. 13 ) is input from theprocessing circuit 262. In step ST70, in a case where the backgroundregion image data 70A2 is not input from the processing circuit 262, anegative determination is made, and the first signal processingtransitions to step ST82. In step ST70, in a case where the backgroundregion image data 70A2 is input from the processing circuit 262, apositive determination is made, and the first signal processingtransitions to step ST72.

In step ST72, the first signal processing portion 250 performs thespecific type of signal processing on the background region image data70A2.

In subsequent step ST74, the first signal processing portion 250determines whether or not the face region image data 70B2 (refer to FIG.13 ) transmitted by executing processing of step ST94 of second signalprocessing illustrated in FIG. 17 is received. In step ST74, in a casewhere the face region image data 70B2 is not received, a negativedetermination is made, and the first signal processing transitions tostep ST80. In step ST74, in a case where the face region image data 70B2is received, a positive determination is made, and the first signalprocessing transitions to step ST76.

In step ST80, the first signal processing portion 250 determines whetheror not the first signal processing finish condition is satisfied. Instep ST80, in a case where the first signal processing finish conditionis not satisfied, a negative determination is made, and the first signalprocessing transitions to step ST74. In step ST80, in a case where thefirst signal processing finish condition is satisfied, a positivedetermination is made, and the first signal processing is finished.

In step ST76, the first signal processing portion 250 generates thecombined image data 272 (refer to FIG. 13 ) by performing the combiningprocessing of combining the background region image data 70A2 obtainedby executing processing of step ST72 with the face region image data70B2 received in step ST74.

In subsequent step ST78, the first signal processing portion 250 outputsthe combined image data 272 obtained by executing processing of stepST76 to the controller 46 (refer to FIG. 13 ) through the communicationline 60 (refer to FIG. 13 ).

In subsequent step ST82, the first signal processing portion 250determines whether or not the first signal processing finish conditionis satisfied. In step ST82, in a case where the first signal processingfinish condition is not satisfied, a negative determination is made, andthe first signal processing transitions to step ST70. In step ST82, in acase where the first signal processing finish condition is satisfied, apositive determination is made, and the first signal processing isfinished.

Next, a flow of second signal processing executed by the second signalprocessing portion 252 will be described with reference to FIG. 17 .

In the second signal processing illustrated in FIG. 17 , in step ST90,the second signal processing portion 252 determines whether or not theface region image data 70B2 (refer to FIG. 13 ) is input from theprocessing circuit 262. In step ST90, in a case where the face regionimage data 70B2 is not input from the processing circuit 262, a negativedetermination is made, and the second signal processing transitions tostep ST96. In step ST90, in a case where the face region image data 70B2is input from the processing circuit 262, a positive determination ismade, and the second signal processing transitions to step ST92.

In step ST92, the second signal processing portion 252 performs thespecific type of signal processing on the face region image data 70B2.In addition, the second signal processing portion 252 executes the faceauthentication on the face region image data 70B2 subjected to thespecific type of signal processing.

In subsequent step ST94, the second signal processing portion 252transmits the face region image data 70B2 obtained by executingprocessing of step ST92 to the first signal processing portion 250through the communication line 58 (refer to FIG. 13 ). The faceauthentication result information is assigned to the face region imagedata 70B2 obtained by executing processing of step ST92.

In subsequent step ST96, the second signal processing portion 252determines whether or not the second signal processing finish conditionis satisfied. In step ST96, in a case where the second signal processingfinish condition is not satisfied, a negative determination is made, andthe second signal processing transitions to step ST90. In step ST96, ina case where the second signal processing finish condition is satisfied,a positive determination is made, and the second signal processing isfinished.

As described above, in the imaging apparatus 200, the captured imagedata 70 is separated into the background region image data 70A2 and theface region image data 70B2. The background region image data 70A2 isoutput to the first signal processing portion 250 through the firstoutput line 53, and the face region image data 70B2 is output to thesecond signal processing portion 252 through the second output line 55.

Generally, the face region image data 70B2 is image data that isprioritized over the background region image data 70A2. Thus, in thesecond signal processing portion 252, more complex processing isperformed on the face region image data 70B2 than on the backgroundregion image data 70A2. In the second embodiment, the faceauthentication is executed as the complex processing. The backgroundregion image data 70A2 subjected to the specific type of signalprocessing and the face region image data 70B2 on which the faceauthentication is executed in addition to the specific type of signalprocessing are combined by the first signal processing portion 250.

Since the first signal processing portion 250 and the second signalprocessing portion 252 execute processing of different degrees ofcomplexity, a load required for image processing can be reduced,compared to a case of constantly executing the complex processing on theentire captured image data 70.

In addition, as illustrated in FIG. 14B, each of the background regionimage 70A2 a and the face region image 70B2 a adjacent in the up-downdirection UD has the overlapping region 271 between the backgroundregion image 70A2 a and the face region image 70B2 a. Accordingly, inthe signal processing circuit 250B, the combined image data 272 in whichnoticeability of a boundary region between the background region image70A2 a and the face region image 70B2 a is suppressed, compared to acase of joining two images obtained by simple division into two parts isgenerated.

While an example of a form of separating the captured image data 70 intothe background region image data 70A2 and the face region image data70B2 is illustratively described in the second embodiment, thetechnology of the present disclosure is not limited thereto. Forexample, image data that is designated in the captured image data 70 bythe user through the reception portion 84 may be output to the secondsignal processing portion 252, and the remaining image data of thecaptured image data 70 may be output to the first signal processingportion 250. For example, the image data designated by the user refersto image data that is defined as important image data in the capturedimage data 70 in accordance with an instruction received by thereception portion 84. For example, the important image data refers toimage data indicating a partial region including an image of a personand/or a building to which the user pays attention.

Even in this case, it is preferable to dispose an overlapping regioncorresponding to the overlapping region 271 between an image indicatedby the image data designated by the user and an image indicated by theremaining image data of the captured image data 70. Accordingly, in thesignal processing circuit 250B, combined image data in whichnoticeability of a boundary region between the image indicated by thedesignated image data and the image indicated by the remaining imagedata is suppressed, compared to a case of joining two images by simpledivision into two parts is generated.

In addition, while the face authentication is illustrated as an exampleof the complex processing in the second embodiment, the technology ofthe present disclosure is not limited thereto. For example, pupildetection processing of detecting a pupil of a person, red eyecorrection processing of correcting a red eye, and/or electronic shakecorrection processing (for example, EIS) is illustrated as the complexprocessing.

Third Embodiment

An example of a form of separating the captured image data 70 into theleft image data 70A1 and the right image data 70B1 is described in thefirst embodiment. An example of a form of compressing the captured imagedata 70 and then, separating the captured image data 70 into two piecesof image data will be described in a third embodiment. In the thirdembodiment, the same constituents as the first embodiment will bedesignated by the same reference signs and will not be described.Hereinafter, parts different from the first embodiment will bedescribed.

As illustrated in FIG. 1 , an imaging apparatus 300 according to thethird embodiment is different from the imaging apparatus 10 described inthe first embodiment in that an imaging apparatus main body 312 isincluded instead of the imaging apparatus main body 12.

The imaging apparatus main body 312 is different from the imagingapparatus main body 12 in that an imaging element 344 (refer to FIG. 18) is included instead of the imaging element 44, and that a first signalprocessing portion 350 (refer to FIG. 18 ) is included instead of thefirst signal processing portion 50. The first signal processing portion350 is different from the first signal processing portion 50 in that asignal processing circuit 350B is included instead of the signalprocessing circuit 50B.

As illustrated in FIG. 18 , the imaging element 344 is different fromthe imaging element 44 in that a processing circuit 362 is includedinstead of the processing circuit 62. The processing circuit 362 isdifferent from the processing circuit 62 in that an image processingcircuit 362C is included instead of the image processing circuit 62C,and that an output circuit 362D is included instead of the outputcircuit 62D.

The image processing circuit 362C acquires the captured image data 70from the memory 64. The captured image data 70 acquired from the memory64 by the image processing circuit 362C is color image data in which Rpixels, G pixels, and B pixels are periodically arranged. As illustratedin FIG. 19 as an example, in the captured image data 70, the R pixels,the G pixels, and the B pixels are arranged with periodicitycorresponding to X-Trans (registered trademark) arrangement. The Rpixels, the G pixels, and the B pixels are an example of a “plurality ofprimary color pixels” according to the embodiments of the technology ofthe present disclosure.

In the example illustrated in FIG. 19 , in a first row, the R pixels,the G pixels, and the B pixels are arranged in circulation in an orderof the G pixel, the B pixel, the R pixel, the G pixel, the R pixel, andthe B pixel in a row direction. In addition, in a second row, the Rpixels, the G pixels, and the B pixels are arranged in circulation in anorder of the R pixel, the G pixel, the G pixel, the B pixel, the Gpixel, and the G pixel in the row direction. In addition, in a thirdrow, the R pixels, the G pixels, and the B pixels are arranged incirculation in an order of the B pixel, the G pixel, the G pixel, the Rpixel, the G pixel, and the G pixel in the row direction. In addition,in a fourth row, the R pixels, the G pixels, and the B pixels arearranged in circulation in an order of the G pixel, the R pixel, the Bpixel, the G pixel, the B pixel, and the R pixel in the row direction.In addition, in a fifth row, the R pixels, the G pixels, and the Bpixels are arranged in circulation in an order of the B pixel, the Gpixel, the G pixel, the R pixel, the G pixel, and the G pixel in the rowdirection. Furthermore, in a sixth row, the R pixels, the G pixels, andthe B pixels are arranged in circulation in an order of the R pixel, theG pixel, the G pixel, the B pixel, the G pixel, and the G pixel in therow direction. An arrangement pattern of the R pixels, the G pixels, andthe B pixels of the entire captured image data 70 is formed byrepetition of an arrangement pattern of the R pixels, the G pixels, andthe B pixels of the first row to the sixth row in units of six rows in acolumn direction.

The image processing circuit 362C compresses the captured image data 70acquired from the memory 64. That is, the image processing circuit 362Cgenerates vertically thinned image data 73 from the captured image data70. The vertically thinned image data 73 is image data obtained bythinning out the captured image data 70 in units of rows. Specifically,as illustrated in FIG. 19 as an example, the vertically thinned imagedata 73 is image data indicating a vertically ½ thinned image obtainedby thinning out pixels of lines of even-numbered rows in the columndirection from the captured image indicated by the captured image data70.

The image processing circuit 362C separates the vertically thinned imagedata 73 into odd-numbered column image data 73A and even-numbered columnimage data 73B and outputs the odd-numbered column image data 73A andthe even-numbered column image data 73B obtained by separation to theoutput circuit 362D. While an example of a form of obtaining theodd-numbered column image data 73A and the even-numbered column imagedata 73B as a plurality of pieces of divided image data by thinning outand then, dividing the captured image data 70 is illustrativelydescribed here, the technology of the present disclosure is not limitedthereto. For example, the plurality of pieces of divided image data maybe obtained by directly dividing the captured image data 70.

As illustrated in FIG. 20 and FIG. 21 as an example, the odd-numberedcolumn image data 73A is image data indicating an odd-numbered columnimage obtained by thinning out pixels of lines of even-numbered columnsfrom the vertically ½ thinned image indicated by the vertically thinnedimage data 73. That is, the odd-numbered column image data 73A is imagedata indicating a set of pixels of odd-numbered columns in thevertically ½ thinned image. In addition, as illustrated in FIG. 20 andFIG. 21 as an example, the even-numbered column image data 73B is imagedata indicating an even-numbered column image obtained by thinning outpixels of lines of odd-numbered columns from the vertically ½ thinnedimage indicated by the vertically thinned image data 73. That is, theeven-numbered column image data 73B is color image data indicating a setof pixels of even-numbered columns in the vertically ½ thinned image. Inother words, each of the odd-numbered column image data 73A and theeven-numbered column image data 73B is color image data indicating animage in which each of the R pixels, the G pixels, and the B pixels isperiodically arranged. The odd-numbered column image data 73A and theeven-numbered column image data 73B are an example of a “plurality ofpieces of primary color pixel arrangement image data” and a “pluralityof pieces of divided image data” according to the embodiments of thetechnology of the present disclosure.

The output circuit 362D outputs the odd-numbered column image data 73Ainput from the image processing circuit 362C to the first signalprocessing portion 350 through the first output line 53. In addition,the output circuit 362D outputs the even-numbered column image data 73Binput from the image processing circuit 362C to the second signalprocessing portion 52 through the second output line 55.

In the second signal processing portion 52, the same processing asprocessing performed on the right image data 70B1 described in the firstembodiment is performed on the even-numbered column image data 73B, andthe even-numbered column image data 73B after processing is transmittedto the first signal processing portion 350.

In the first signal processing portion 350, the reception circuit 50Creceives the even-numbered column image data 73B transmitted from thesecond signal processing portion 52. The signal processing circuit 350Bacquires the even-numbered column image data 73B received by thereception circuit 50C.

Meanwhile, the odd-numbered column image data 73A is input into thebuffer 50A. The buffer 50A temporarily holds the odd-numbered columnimage data 73A and outputs the odd-numbered column image data 73A to thesignal processing circuit 350B using the FIFO method. The signalprocessing circuit 350B performs the specific type of signal processingon the odd-numbered column image data 73A input from the buffer 50A. Inaddition, the signal processing circuit 350B generates combined imagedata 372 by combining the odd-numbered column image data 73A subjectedto the specific type of signal processing with the even-numbered columnimage data 73B acquired from the reception circuit 50C. Consequently, anarrangement pattern of R pixels, G pixels, and B pixels of an imageindicated by the combined image data 372 is the same arrangement patternas the vertically ½ thinned image. That is, the arrangement pattern ofthe R pixels, the G pixels, and the B pixels of the image indicated bythe combined image data 372 is a periodic arrangement pattern in whichthe demosaicing can be performed on the R pixels, the G pixels, and theB pixels.

Therefore, the signal processing circuit 350B performs the demosaicingof the R, G, and B signals on the combined image data 372 and outputsthe combined image data 372 subjected to the demosaicing to thecontroller 46 through the communication line 60.

Next, an action of the imaging apparatus 300 will be described.

First, a flow of imaging processing executed by the processing circuit362 of the imaging element 344 will be described with reference to FIG.22 . The imaging processing illustrated in FIG. 22 is different from theimaging processing illustrated in FIG. 10 in that processing of stepST100 is included instead of processing of step ST18, and that stepST102 is included instead of processing of step ST20. Thus, in aflowchart of the imaging processing illustrated in FIG. 22 , the samesteps as the imaging processing illustrated in FIG. 10 are designated bythe same step numbers. Hereinafter, only parts of the imaging processingillustrated in FIG. 22 different from the imaging processing illustratedin FIG. 10 will be described.

In the imaging processing illustrated in FIG. 22 , in step ST100, theimage processing circuit 362C generates the vertically thinned imagedata 73 (refer to FIG. 18 to FIG. 21 ) from the captured image data 70.The image processing circuit 362C generates the odd-numbered columnimage data 73A and the even-numbered column image data 73B from thegenerated vertically thinned image data 73. That is, the verticallythinned image data 73 is separated into the odd-numbered column imagedata 73A and the even-numbered column image data 73B (refer to FIG. 20and FIG. 21 ).

In subsequent step ST102, the output circuit 362D outputs theodd-numbered column image data 73A to the first signal processingportion 350 through the first output line 53. In addition, the outputcircuit 362D outputs the even-numbered column image data 73B to thesecond signal processing portion 52 through the second output line 55.

Next, a flow of first signal processing executed by the first signalprocessing portion 350 will be described with reference to FIG. 23 .

In the first signal processing illustrated in FIG. 23 , in step ST210,the first signal processing portion 350 determines whether or not theodd-numbered column image data 73A (refer to FIG. 18 ) is input from theprocessing circuit 362. In step ST210, in a case where the odd-numberedcolumn image data 73A is not input from the processing circuit 362, anegative determination is made, and the first signal processingtransitions to step ST222. In step ST210, in a case where theodd-numbered column image data 73A is input from the processing circuit362, a positive determination is made, and the first signal processingtransitions to step ST212.

In step ST212, the first signal processing portion 350 performs thespecific type of signal processing on the odd-numbered column image data73A.

In subsequent step ST214, the first signal processing portion 350determines whether or not the even-numbered column image data 73B (referto FIG. 18 ) transmitted by executing processing of step ST234 of secondsignal processing illustrated in FIG. 24 is received. In step ST214, ina case where the even-numbered column image data 73B is not received, anegative determination is made, and the first signal processingtransitions to step ST220. In step ST214, in a case where theeven-numbered column image data 73B is received, a positivedetermination is made, and the first signal processing transitions tostep ST216.

In step ST220, the first signal processing portion 350 determineswhether or not the first signal processing finish condition issatisfied. In step ST220, in a case where the first signal processingfinish condition is not satisfied, a negative determination is made, andthe first signal processing transitions to step ST214. In step ST220, ina case where the first signal processing finish condition is satisfied,a positive determination is made, and the first signal processing isfinished.

In step ST216, the first signal processing portion 350 generates thecombined image data 372 (refer to FIG. 18 ) by performing the combiningprocessing of combining the odd-numbered column image data 73A obtainedby executing processing of step ST212 with the even-numbered columnimage data 73B received in step ST214. The first signal processingportion 350 performs the demosaicing on the combined image data 372.

In subsequent step ST218, the first signal processing portion 350outputs the combined image data 372 obtained by executing processing ofstep ST216 to the controller 46 (refer to FIG. 18 ) through thecommunication line 60 (refer to FIG. 18 ).

In subsequent step ST222, the first signal processing portion 350determines whether or not the first signal processing finish conditionis satisfied. In step ST222, in a case where the first signal processingfinish condition is not satisfied, a negative determination is made, andthe first signal processing transitions to step ST210. In step ST222, ina case where the first signal processing finish condition is satisfied,a positive determination is made, and the first signal processing isfinished.

Next, a flow of second signal processing executed by the second signalprocessing portion 52 will be described with reference to FIG. 24 .

In the second signal processing illustrated in FIG. 24 , in step ST230,the second signal processing portion 52 determines whether or not theeven-numbered column image data 73B (refer to FIG. 18 ) is input fromthe processing circuit 362. In step ST230, in a case where theeven-numbered column image data 73B is not input from the processingcircuit 362, a negative determination is made, and the second signalprocessing transitions to step ST236. In step ST230, in a case where theeven-numbered column image data 73B is input from the processing circuit362, a positive determination is made, and the second signal processingtransitions to step ST232.

In step ST232, the second signal processing portion 52 performs thespecific type of signal processing on the even-numbered column imagedata 73B.

In subsequent step ST234, the second signal processing portion 52transmits the even-numbered column image data 73B obtained by executingprocessing of step ST232 to the first signal processing portion 350through the communication line 58 (refer to FIG. 18 ).

In subsequent step ST236, the second signal processing portion 52determines whether or not the second signal processing finish conditionis satisfied. In step ST236, in a case where the second signalprocessing finish condition is not satisfied, a negative determinationis made, and the second signal processing transitions to step ST230. Instep ST236, in a case where the second signal processing finishcondition is satisfied, a positive determination is made, and the secondsignal processing is finished.

As described above, in the third embodiment, the captured image data 70is color image data indicating a color captured image in which the Rpixels, the G pixels, and the B pixels which are the plurality ofprimary color pixels are periodically arranged. In addition, thecaptured image data 70 is divided into the plurality of pieces ofprimary color pixel arrangement image data as a plurality of pieces ofimage data. In the examples illustrated in FIG. 18 and FIG. 20 , theodd-numbered column image data 73A and the even-numbered column imagedata 73B are illustrated as the plurality of pieces of primary colorpixel arrangement image data. In addition, each of the odd-numberedcolumn image data 73A and the even-numbered column image data 73B isimage data indicating an image in which each of the R pixels, the Gpixels, and the B pixels is periodically arranged. By using the imagedata indicating the image in which each of the R pixels, the G pixels,and the B pixels is periodically arranged like the odd-numbered columnimage data 73A and the even-numbered column image data 73B, thedemosaicing for the R pixels, the G pixels, and the B pixels can beimplemented.

In addition, as illustrated in FIG. 18 and FIG. 19 , the odd-numberedcolumn image data 73A and the even-numbered column image data 73B areimage data obtained by division from the vertically thinned image data73. The vertically thinned image data 73 has a smaller data amount thanthe captured image data 70. Accordingly, the specific type of signalprocessing for the odd-numbered column image data 73A and theeven-numbered column image data 73B can be performed at high speed,compared to a case of performing the specific type of signal processingon one of two pieces of image data obtained by dividing the capturedimage data 70 without thinning.

In addition, the odd-numbered column image data 73A is image dataindicating the set of the pixels of the odd-numbered columns in thevertically ½ thinned image, and the even-numbered column image data 73Bis image data indicating the set of the pixels of the even-numberedcolumns in the vertically ½ thinned image. Processing contents of thespecific type of signal processing and combining for a plurality ofpieces of image data obtained by irregularly dividing the vertically ½thinned image are more complex than processing for the odd-numberedcolumn image data 73A and the even-numbered column image data 73B.

Accordingly, high-speed processing can be implemented for theodd-numbered column image data 73A and the even-numbered column imagedata 73B, compared to processing for the plurality of pieces of imagedata obtained by irregularly dividing the vertically ½ thinned image.

In addition, the first signal processing portion 350 performs thedemosaicing on the combined image data 372 obtained by combining theodd-numbered column image data 73A with the even-numbered column imagedata 73B. Thus, a high image quality image can be obtained, compared toa case of not performing the demosaicing.

While an example of a form in which the first signal processing portion350 uses the DRAM 54, and in which the second signal processing portion52 uses the DRAM 56 is illustratively described in the third embodiment,the technology of the present disclosure is not limited thereto. Forexample, as illustrated in FIG. 25 , without using the DRAMs 54 and 56,a first signal processing portion 750 may be applied instead of thefirst signal processing portion 350, and a second signal processingportion 752 may be applied instead of the second signal processingportion 52. In the example illustrated in FIG. 25 , the first signalprocessing portion 750 is different from the first signal processingportion 350 in that a line memory 750A is included, and that the DRAM 54is not used. In addition, the second signal processing portion 752 isdifferent from the second signal processing portion 52 in that a linememory 752A is included, and that the DRAM 56 is not used.

In the first signal processing portion 750, the line memory 750A isinterposed between the buffer 50A and the signal processing circuit350B. The buffer 50A outputs the odd-numbered column image data 73A tothe line memory 750A. The line memory 750A stores the odd-numberedcolumn image data 73A input from the buffer 50A in units of lines andoutputs the odd-numbered column image data 73A to the signal processingcircuit 350B using the FIFO method. The signal processing circuit 350Bexecutes processing described in the third embodiment.

Meanwhile, in the second signal processing portion 752, the line memory752A is interposed between the buffer 52A and the signal processingcircuit 52B. The buffer 52A outputs the even-numbered column image data73B to the line memory 752A. The line memory 752A stores theeven-numbered column image data 73B input from the buffer 52A in unitsof lines and outputs the even-numbered column image data 73B to thesignal processing circuit 52B using the FIFO method. The signalprocessing circuit 52B executes processing described in the thirdembodiment.

In addition, while the image data indicating the vertically ½ thinnedimage is illustrated as compressed image data in the third embodiment,the technology of the present disclosure is not limited thereto. Forexample, in a case where n denotes a natural number greater than orequal to 3, image data indicating a vertically 1/n thinned image may beapplied as the compressed image data. In addition, image data indicatinga horizontally thinned image that is thinned in units of columns may beapplied as the compressed image data, or image data indicating an imagethat is thinned in units of rows and units of columns may be applied asthe compressed image data.

Fourth Embodiment

An example of a form of separating the captured image data 70 into theleft image data 70A1 and the right image data 70B1 is described in thefirst embodiment. An example of a form of setting a separation methodfor the captured image data 70 to vary depending on the operation modewill be described in a fourth embodiment. In the fourth embodiment, thesame constituents as the first embodiment will be designated by the samereference signs and will not be described. Hereinafter, parts differentfrom the first embodiment will be described.

As illustrated in FIG. 1 , an imaging apparatus 400 according to thefourth embodiment is different from the imaging apparatus 10 describedin the first embodiment in that an imaging apparatus main body 412 isincluded instead of the imaging apparatus main body 12.

The imaging apparatus main body 412 is different from the imagingapparatus main body 12 in that an imaging element 444 (refer to FIG. 26) is included instead of the imaging element 44. The imaging element 444is different from the imaging element 44 in that a processing circuit462 is included instead of the processing circuit 62. The processingcircuit 462 is different from the processing circuit 62 in that an imageprocessing circuit 462C is included instead of the image processingcircuit 62C, and that an output circuit 462D is included instead of theoutput circuit 62D.

The imaging apparatus main body 412 is different from the imagingapparatus main body 12 in that a first signal processing portion 450(refer to FIG. 26 ) is included instead of the first signal processingportion 50, and that a second signal processing portion 452 (refer toFIG. 26 ) is included instead of the second signal processing portion52.

The first signal processing portion 450 is different from the firstsignal processing portion 50 in that a function of the first signalprocessing portion 50 and a function of the first signal processingportion 350 described in the third embodiment are included. In addition,the first signal processing portion 450 is different from the firstsignal processing portion 50 in that the function of the first signalprocessing portion 50 and the function of the first signal processingportion 350 are selectively operated.

The second signal processing portion 452 is different from the secondsignal processing portion 52 in that a function of the second signalprocessing portion 52 and a function of the second signal processingportion 252 described in the second embodiment are included. Inaddition, the second signal processing portion 452 is different from thesecond signal processing portion 52 in that the function of the secondsignal processing portion 52 and the function of the second signalprocessing portion 252 described in the second embodiment areselectively operated.

The controller 46 selectively outputs a still picture image capturingmode signal 480A (refer to FIG. 26 ) and a display motion picturecapturing mode signal 480B (refer to FIG. 27 ) to the processing circuit462 through the communication line 60. For example, the still pictureimage capturing mode signal 480A is output from the controller 46 in acase where an instruction to set the imaging apparatus 400 to the stillpicture image capturing mode is received by the reception portion 84. Inaddition, for example, the display motion picture capturing mode signal480B is output from the controller 46 in a case where an instruction toset the imaging apparatus 400 to the display motion picture capturingmode is received by the reception portion 84.

As illustrated in FIG. 26 , in a case where the still picture imagecapturing mode signal 480A is input from the controller 46 through thecommunication line 60, the processing circuit 462 operates the imageprocessing circuit 462C in the same manner as the image processingcircuit 62C described in the first embodiment. That is, the processingcircuit 462 causes the image processing circuit 462C to separate thecaptured image data 70 into the left image data 70A1 and the right imagedata 70B1 and output the left image data 70A1 and the right image data70B1 to the output circuit 462D. The left image 70A1 a indicated by theleft image data 70A1 and the right image 70B1 a indicated by the rightimage data 70B1 have the overlapping region 71 described in the firstembodiment. The left image data 70A1 and the right image data 70B1according to the fourth embodiment are an example of a “plurality ofoverlapping image data” according to the embodiments of the technologyof the present disclosure.

As illustrated in FIG. 26 , the output circuit 462D outputs the leftimage data 70A1 to the first signal processing portion 450 through thefirst output line 53 in the same manner as the output circuit 62Ddescribed in the first embodiment. In addition, the output circuit 462Doutputs the right image data 70B1 to the second signal processingportion 452 through the second output line 55 in the same manner as theoutput circuit 62D described in the first embodiment.

The first signal processing portion 450 performs the specific type ofsignal processing on the input left image data 70A1. The second signalprocessing portion 452 performs the specific type of signal processingon the input right image data 70B1 and outputs the right image data 70B1subjected to the specific type of signal processing to the first signalprocessing portion 450.

The first signal processing portion 450 generates the combined imagedata 72 described in the first embodiment by combining the left imagedata 70A1 subjected to the specific type of signal processing with theright image data 70B1 input from the second signal processing portion452. The first signal processing portion 450 outputs the generatedcombined image data 72 to the controller 46.

Meanwhile, as illustrated in FIG. 27 , in a case where the displaymotion picture capturing mode signal 480B is input from the controller46 through the communication line 60, the processing circuit 462operates the image processing circuit 462C in the same manner as theimage processing circuit 362C described in the third embodiment. Thatis, the processing circuit 462 causes the image processing circuit 462Cto compress the captured image data 70 into the vertically thinned imagedata 73 and then, separate the vertically thinned image data 73 into theodd-numbered column image data 73A and the even-numbered column imagedata 73B. The processing circuit 462 causes the image processing circuit462C to output the odd-numbered column image data 73A and theeven-numbered column image data 73B to the output circuit 462D.

As illustrated in FIG. 27 , the output circuit 462D outputs theodd-numbered column image data 73A to the first signal processingportion 450 through the first output line 53 in the same manner as theoutput circuit 362D described in the third embodiment. In addition, theoutput circuit 462D outputs the even-numbered column image data 73B tothe second signal processing portion 452 through the second output line55 in the same manner as the output circuit 362D described in the thirdembodiment.

The first signal processing portion 450 performs the specific type ofsignal processing on the input odd-numbered column image data 73A. Thesecond signal processing portion 452 performs the specific type ofsignal processing on the input even-numbered column image data 73B andoutputs the even-numbered column image data 73B subjected to thespecific type of signal processing to the first signal processingportion 450.

The first signal processing portion 450 generates the combined imagedata 372 described in the third embodiment by combining the odd-numberedcolumn image data 73A subjected to the specific type of signalprocessing with the even-numbered column image data 73B input from thesecond signal processing portion 452. The first signal processingportion 450 outputs the generated combined image data 372 to thecontroller 46.

Next, an action of the imaging apparatus 400 will be described.

First, a flow of imaging processing executed by the processing circuit462 of the imaging element 444 will be described with reference to FIG.28 . Here, for convenience of description, it is assumed that theimaging apparatus 400 is set to the still picture image capturing modeor the display motion picture capturing mode.

In the imaging processing illustrated in FIG. 28 , first, in step ST300,the image processing circuit 462C determines whether or not the imagingapparatus 400 is in the still picture image capturing mode. In stepST300, in a case where the imaging apparatus 400 is not in the stillpicture image capturing mode, that is, in a case where the imagingapparatus 400 is in the display motion picture capturing mode, anegative determination is made, and the imaging processing transitionsto step ST304. In step ST300, in a case where the imaging apparatus 400is in the still picture image capturing mode, a positive determinationis made, and the imaging processing transitions to step ST302.

In step ST302, the processing circuit 462 executes still picture imagecapturing processing. Then, the imaging processing transitions to stepST306. The still picture image capturing processing is the sameprocessing as the imaging processing (refer to FIG. 10 ) described inthe first embodiment.

In step ST304, the processing circuit 462 executes display motionpicture image capturing processing. Then, the imaging processingtransitions to step ST306. The display motion picture imaging processingis the same processing as the imaging processing (refer to FIG. 22 )described in the third embodiment.

In step ST306, the processing circuit 462 determines whether or not theimaging processing finish condition is satisfied. In step ST306, in acase where the imaging processing finish condition is not satisfied, anegative determination is made, and the imaging processing transitionsto step ST300. In step ST306, in a case where the imaging processingfinish condition is satisfied, a positive determination is made, and theimaging processing is finished.

In the still picture image capturing mode, in the first signalprocessing portion 450, the same processing as the first signalprocessing (refer to FIG. 11 ) described in the first embodiment isexecuted. In addition, in the still picture image capturing mode, in thesecond signal processing portion 452, the same processing as the secondsignal processing (refer to FIG. 12 ) described in the first embodimentis executed.

Meanwhile, in the display motion picture capturing mode, in the firstsignal processing portion 450, the same processing as the first signalprocessing (FIG. 23 ) described in the third embodiment is executed. Inaddition, in the display motion picture capturing mode, in the secondsignal processing portion 452, the same processing as the second signalprocessing (FIG. 24 ) described in the third embodiment is executed.

As described above, in the fourth embodiment, in the still picture imagecapturing mode, the captured image data 70 is separated into the leftimage data 70A1 and the right image data 70B1 described in the firstembodiment. In addition, in the display motion picture capturing mode,the captured image data 70 is separated into the odd-numbered columnimage data 73A and the even-numbered column image data 73B described inthe third embodiment. Accordingly, a balance among image quality, powerconsumption, and a processing speed can be set to vary between the stillpicture image capturing mode and the display motion picture capturingmode.

In addition, in the still picture image capturing mode, the capturedimage data 70 is separated into the left image data 70A1 and the rightimage data 70B1 each of which includes the image data indicating theoverlapping region 71. Meanwhile, in the display motion picturecapturing mode, the captured image data 70 is separated in units oflines. Accordingly, in the still picture image capturing mode, sinceprocessing is performed on the left image data 70A1 and the right imagedata 70B1 each of which includes the image data indicating theoverlapping region 71, the image quality can be increased, compared tothe display motion picture capturing mode. In addition, in the displaymotion picture capturing mode, processing is performed on theodd-numbered column image data 73A and the even-numbered column imagedata 73B having a smaller data amount than the left image data 70A1 andthe right image data 70B1. Thus, in the display motion picture capturingmode, the power consumption can be reduced, and the processing speed canbe increased, compared to the still picture image capturing mode.

While a case of applying the imaging processing (refer to FIG. 10 )described in the first embodiment as processing of step ST302 (refer toFIG. 28 ) is described in the third embodiment, the technology of thepresent disclosure is not limited thereto. For example, the imagingprocessing (refer to FIG. 15 ) described in the second embodiment may beapplied as processing of step ST302 (refer to FIG. 28 ). In this case,in the still picture image capturing mode, the first signal processing(refer to FIG. 16 ) described in the second embodiment is executed inthe first signal processing portion 450, and the second signalprocessing (refer to FIG. 17 ) described in the second embodiment isexecuted in the second signal processing portion 452.

In addition, while an example of a form of separating the captured imagedata 70 into the left image data 70A1 and the right image data 70B1 inthe still picture image capturing mode is illustratively described inthe third embodiment, the technology of the present disclosure is notlimited thereto. For example, in the still picture image capturing mode,as described in the second embodiment, the captured image data 70 may beseparated into the background region image data 70A2 and the face regionimage data 70B2. In this case, in the still picture image capturingmode, the first signal processing portion 450 may be operated in thesame manner as the first signal processing portion 250 described in thesecond embodiment, and the second signal processing portion 452 may beoperated in the same manner as the second signal processing portion 252described in the second embodiment.

Fifth Embodiment

An example of a form of compressing the captured image data 70 into theodd-numbered column image data 73A and the even-numbered column imagedata 73B is described in the third embodiment. An example of a form ofcompressing the captured image data 70 into two pieces of image datausing another method will be described in a fifth embodiment. In thefifth embodiment, the same constituents as the third embodiment will bedesignated by the same reference signs and will not be described.Hereinafter, parts different from the third embodiment will bedescribed.

As illustrated in FIG. 1 , an imaging apparatus 500 according to thefifth embodiment is different from the imaging apparatus 300 describedin the third embodiment in that an imaging apparatus main body 512 isincluded instead of the imaging apparatus main body 312.

The imaging apparatus main body 512 is different from the imagingapparatus main body 312 in that an imaging element 544 (refer to FIG. 29) is included instead of the imaging element 344. The imaging element544 is different from the imaging element 344 in that a processingcircuit 562 is included instead of the processing circuit 362. Theprocessing circuit 562 is different from the processing circuit 362 inthat an image processing circuit 562C is included instead of the imageprocessing circuit 362C, and that an output circuit 562D is includedinstead of the output circuit 362D.

The imaging apparatus main body 512 is different from the imagingapparatus main body 312 in that a first signal processing portion 550(refer to FIG. 29 ) is included instead of the first signal processingportion 50.

The memory 64 stores captured image data 570. The image processingcircuit 562C acquires the captured image data 570 from the memory 64.The captured image data 570 is color image data including R pixels, Gpixels, and B pixels. As illustrated in FIG. 30 as an example, in thecaptured image data 570, the R pixels, the G pixels, and the B pixelsare arranged with periodicity corresponding to Bayer arrangement.

In the example illustrated in FIG. 30 , in a first row, the R pixels andthe G pixels are arranged in circulation in an order of the R pixel andthe G pixel in the row direction. In addition, in a second row, the Bpixels and the G pixels are arranged in circulation in an order of the Gpixel and the B pixel in the row direction. An arrangement pattern ofthe R pixels, the G pixels, and the B pixels of the entire capturedimage data 570 is formed by repetition of an arrangement pattern of theR pixels and the G pixels of the first row in every other row in thecolumn direction and repetition of an arrangement pattern of the Bpixels and the G pixels of the second row in every other row in thecolumn direction.

The image processing circuit 562C compresses the captured image data 570acquired from the memory 64. That is, the image processing circuit 562Cgenerates vertically thinned image data 573 from the captured image data570. As illustrated in FIG. 30 as an example, the vertically thinnedimage data 573 is image data indicating a vertically ½ thinned imageobtained by thinning out lines of every two rows adjacent in the columndirection from a captured image indicated by the captured image data570.

As illustrated in FIG. 29 and FIG. 31 as an example, the imageprocessing circuit 562C separates the vertically thinned image data 753into first horizontally thinned image data 573A and second horizontallythinned image data 573B. The image processing circuit 562C outputs thefirst horizontally thinned image data 573A and the second horizontallythinned image data 573B obtained by separation to the output circuit562D.

As illustrated in FIG. 31 as an example, the first horizontally thinnedimage data 573A is image data indicating one of a pair of horizontally ½thinned images that are obtained by thinning out every two columnsdifferent between the horizontally ½ thinned images in the row directionin units of two columns from the vertically ½ thinned image indicated bythe vertically thinned image data 573. In addition, the secondhorizontally thinned image data 573B is image data indicating the otherof the pair of horizontally ½ thinned images.

The output circuit 562D outputs the first horizontally thinned imagedata 573A input from the image processing circuit 562C to the firstsignal processing portion 550 through the first output line 53. Inaddition, the output circuit 562D outputs the second horizontallythinned image data 573B input from the image processing circuit 562C tothe second signal processing portion 52 through the second output line55.

In the second signal processing portion 52, the same processing asprocessing performed on the even-numbered column image data 73Bdescribed in the third embodiment is performed on the secondhorizontally thinned image data 573B, and the second horizontallythinned image data 573B after processing is transmitted to the firstsignal processing portion 550.

The first signal processing portion 550 receives the second horizontallythinned image data 573B transmitted from the second signal processingportion 52.

In the first signal processing portion 550, the same processing asprocessing performed on the odd-numbered column image data 73A describedin the third embodiment is performed on the first horizontally thinnedimage data 573A. In the first signal processing portion 550, combinedimage data 572 is generated by combining the first horizontally thinnedimage data 573A with the second horizontally thinned image data 573B.Consequently, an arrangement pattern of R pixels, G pixels, and B pixelsof an image indicated by the combined image data 572 is the samearrangement pattern as the vertically ½ thinned image indicated by thevertically thinned image data 573. That is, the arrangement pattern ofthe R pixels, the G pixels, and the B pixels of the image indicated bythe combined image data 572 is a periodic arrangement pattern in whichthe demosaicing can be performed on the R pixels, the G pixels, and theB pixels.

Therefore, the first signal processing portion 550 performs thedemosaicing of the R, G, and B signals on the combined image data 572and outputs the combined image data 572 subjected to the demosaicing tothe controller 46 through the communication line 60 in the same manneras the third embodiment.

As described above, the Bayer arrangement is employed in the capturedimage data 570. Even in this case, in the same manner as theodd-numbered column image data 73A and the even-numbered column imagedata 73B described in the third embodiment, the first horizontallythinned image data 573A and the second horizontally thinned image data573B are obtained as two pieces of image data on which the demosaicingcan be performed. Thus, even in a case where the captured image data 570is image data having the Bayer arrangement, the same effect as the thirdembodiment can be obtained.

While the first horizontally thinned image data 573A and the secondhorizontally thinned image data 573B are illustrated as two pieces ofimage data into which the captured image data 570 is compressed in thefifth embodiment, the technology of the present disclosure is notlimited thereto. For example, as illustrated in FIG. 32 , the imageprocessing circuit 562C may separate the captured image data 570 intohigh-order bit data 570C and low-order bit data 570D. In a case wherethe number of bits for each pixel of the captured image data 570 is 16bits, the high-order bit data 570C is, for example, image data in whichthe number of bits for each pixel is high-order 8 bits, and thelow-order bit data 570D is, for example, image data in which the numberof bits for each pixel is low-order 8 bits.

As illustrated in FIG. 32 , the output circuit 562D outputs thehigh-order bit data 570C to the first signal processing portion 550through the first output line 53 and outputs the low-order bit data 570Dto the second signal processing portion 52 through the second outputline 55. In addition, the second signal processing portion 52 performsthe specific type of signal processing on the low-order bit data 570Dand then, transmits the low-order bit data 570D to the first signalprocessing portion 550 through the communication line 58 in the samemanner as the fifth embodiment. In addition, the first signal processingportion 550 receives the low-order bit data 570D transmitted from thesecond signal processing portion 52 and performs the specific type ofsignal processing on the high-order bit data 570C in the same manner asthe fifth embodiment. The first signal processing portion 550 generatescombined image data 572A by combining the high-order bit data 570Csubjected to the specific type of signal processing with the receivedlow-order bit data 570D in the same manner as the fifth embodiment. Thefirst signal processing portion 550 outputs the generated combined imagedata 572A to the controller 46.

While a case where the number of bits for each pixel of the capturedimage data 570 is 16 bits is illustrated here, the technology of thepresent disclosure is not limited thereto. Image data in which thenumber of bits for each pixel is less than 16 bits may be used, or imagedata having the number of bits such that the number of bits for eachpixel exceeds 16 bits may be used. In addition, for example, a divisionmethod for high-order bits and low-order bits may be a division methodthat is determined in accordance with an application and/orspecifications.

In addition, while an example of separating the captured image data 570into the high-order bit data 570C and the low-order bit data 570D by theimage processing circuit 562C is illustratively described here, thetechnology of the present disclosure is not limited thereto. Forexample, the captured image data 570 may be separated into high-orderbit image data, middle-order bit image data, and low-order bit imagedata. The high-order bit image data, the middle-order bit image data,and the low-order bit image data refer to three pieces of compressedimage data obtained by compressing the captured image data 570 bydividing the captured image data 570 into three bit ranges. In addition,the captured image data 570 may be compressed by dividing the capturedimage data 570 into four or more bit ranges. A plurality of pieces ofcompressed image data may be obtained by dividing the captured imagedata 570 into a plurality of bit ranges.

The captured image data 570 is separated into the plurality of pieces ofcompressed image data (in the example illustrated in FIG. 32 , thehigh-order bit data 570C and the low-order bit data 570D) obtained bydividing the captured image data 570 into the plurality of bit ranges.Accordingly, each of the first signal processing portion 550 and thesecond signal processing portion 52 can perform high-speed processing,compared to a case of processing image data obtained by irregulardivision.

In addition, as illustrated in FIG. 32 , by separating the capturedimage data 570 into the high-order bit data 570C and the low-order bitdata 570D, high-accuracy processing can be performed on the high-orderbit data 570C, compared to the low-order bit data 570D. Meanwhile, forthe low-order bit data 570D, the power consumption can be reduced, andthe processing speed can be increased, compared to the high-order bitdata 570C.

In addition, while an example of a form of separating the captured imagedata 70 and 570 (hereinafter, simply referred to as the “captured imagedata”) into two pieces of image data is illustratively described in eachof the embodiments, the technology of the present disclosure is notlimited thereto. For example, in a case where N denotes a natural numbergreater than or equal to 2, the captured image data may be separatedinto N pieces of image data. As a separation method, for example, amethod of separating the captured image data into N bit ranges isconsidered in addition to a method of dividing the captured image datainto N equal parts.

In a case of separating the captured image data into N pieces of imagedata, for example, as illustrated in FIG. 33 , each of a first signalprocessing portion 650 to an N-th signal processing portion 650N isconnected to the imaging element 44 through a corresponding output line.In the example illustrated in FIG. 33 , the first signal processingportion 650 is connected to the imaging element 44 through the firstoutput line 53, and the N-th signal processing portion 650N is connectedto the imaging element 44 through an N-th output line 53N. The N-thsignal processing portion 650N may transmit image data after signalprocessing to the first signal processing portion 650, and the firstsignal processing portion 650 may combine the N pieces of image data andoutput combined image data obtained by combining to the controller 46.

In addition, while an example of a form of implementing the processingcircuits 62, 262, 362, 462, and 562 (hereinafter, simply referred to asthe “processing circuit”) by an ASIC is illustratively described in eachof the embodiments, the technology of the present disclosure is notlimited thereto. For example, the imaging processing may be implementedby a software configuration based on a computer.

In this case, for example, as illustrated in FIG. 34 , an imagingprocessing program 802 causing a computer 652 incorporated in theimaging elements 44, 244, 344, 444, and 544 to execute the imagingprocessing is stored in a storage medium 800. The computer 652 comprisesa CPU 652A, a ROM 652B, and a RAM 652C. The imaging processing program802 of the storage medium 800 is installed on the computer 652, and theCPU 652A of the computer 652 executes the imaging processing inaccordance with the imaging processing program 802. While a single CPUis illustrated as the CPU 652A here, the technology of the presentdisclosure is not limited thereto. A plurality of CPUs may be employedinstead of the CPU 652A. Any portable storage medium such as an SSD or aUSB memory is illustrated as an example of the storage medium 800.

While the imaging processing program 802 is stored in the storage medium800 in the example illustrated in FIG. 34 , the technology of thepresent disclosure is not limited thereto. For example, the imagingprocessing program 802 may be stored in advance in the ROM 652B, and theCPU 652A may read out the imaging processing program 802 from the ROM652B, load the imaging processing program 802 into the RAM 652C, andexecute the loaded imaging processing program 802.

In addition, the imaging processing program 802 may be stored in astorage portion of another computer, a server apparatus, or the likeconnected to the computer 652 through a communication network (notillustrated), and the imaging processing program 802 may be downloadedto the computer 652 in accordance in response to a request from animaging apparatus 700 having the same configuration as any of theimaging apparatuses 10, 200, 300, 400, and 500. In this case, thedownloaded imaging processing program 802 is executed by the CPU 652A ofthe computer 652.

In addition, the computer 652 may be disposed outside the imagingelements 44, 244, 344, 444, and 544. In this case, the computer 652 maycontrol the processing circuit in accordance with the imaging processingprogram 802.

Various processors illustrated below can be used as a hardware resourcefor executing the imaging processing described in each of theembodiments. For example, as described above, a CPU that is ageneral-purpose processor functioning as the hardware resource forexecuting the imaging processing by executing software, that is, theprogram, is illustrated as a processor. In addition, a dedicatedelectric circuit such as an FPGA, a PLD, or an ASIC that is a processorhaving a circuit configuration dedicatedly designed to execute aspecific type of processing is illustrated as a processor.

The hardware resource for executing the imaging processing may beconfigured with one of those various processors or may be configuredwith a combination of two or more processors of the same type ordifferent types (for example, a combination of a plurality of FPGAs or acombination of a CPU and an FPGA). In addition, the hardware resourcefor executing various types of processing according to the embodimentsof the technology of the present disclosure may be one processor.

As an example of a configuration with one processor, first, asrepresented by a computer such as a client and a server, a form in whichone processor is configured with a combination of one or more CPUs andsoftware, and in which this processor functions as the hardware resourcefor executing the in-imaging element processing is available. Second, asrepresented by a system-on-a-chip (SoC) or the like, a form of using aprocessor that implements, by one IC chip, a function of the entiresystem including a plurality of hardware resources for executing theimaging processing is available. The in-imaging element processing isimplemented using one or more of the various processors as the hardwareresource.

Furthermore, as a hardware structure of those various processors, morespecifically, an electric circuit in which circuit elements such assemiconductor elements are combined can be used.

While an interchangeable lens camera is illustrated as the imagingapparatus in each of the embodiments, the technology of the presentdisclosure is not limited thereto. For example, the technology of thepresent disclosure may be applied to a smart device 900 illustrated inFIG. 35. The smart device 900 illustrated in FIG. 35 as an example is anexample of the imaging apparatus according to the embodiments of thetechnology of the present disclosure. The imaging elements 44, 244, 344,444, and 544 described in the embodiments are mounted in the smartdevice 900. Even with the smart device 900 configured in such a manner,the same actions and effects as the imaging apparatus described in theembodiments are obtained. The technology of the present disclosure canbe applied to not only the smart device 900 but also a personal computeror a wearable terminal apparatus.

While the first display 32 and the second display 86 are illustrated ineach of the embodiments, the technology of the present disclosure is notlimited thereto. For example, a separate display that is retrofit intothe imaging apparatus main body 12 may be used as the “display portion”according to the embodiments of the technology of the presentdisclosure.

In addition, the imaging processing, the first signal processing, andthe second signal processing are merely an example. Accordingly,unnecessary steps may be deleted, new steps may be added, or aprocessing order may be rearranged without departing from a gist of thepresent disclosure.

Above described contents and illustrated contents are detaileddescription for parts according to the embodiment of the technology ofthe present disclosure and are merely one example of the technology ofthe present disclosure. For example, description related to the aboveconfigurations, functions, actions, and effects is description relatedto one example of configurations, functions, actions, and effects of theparts according to the embodiment of the technology of the presentdisclosure. Thus, unnecessary parts may be removed, new elements may beadded, or parts may be replaced in the above described contents and theillustrated contents without departing from the gist of the technologyof the present disclosure. In addition, particularly, descriptionrelated to common technical knowledge or the like that does not need tobe described in terms of embodying the technology of the presentdisclosure is omitted in the above described contents and theillustrated contents in order to avoid complication and facilitateunderstanding of the parts according to the embodiment of the technologyof the present disclosure.

In the present specification, “A and/or B” has the same meaning as “atleast one of A or B”. This means that “A and/or B” may be only A, onlyB, or a combination of A and B. In addition, in the presentspecification, the same approach as “A and/or B” is applied to a casewhere three or more matters are represented by connecting the matterswith “and/or”.

All documents, patent applications, and technical standards disclosed inthe present specification are incorporated in the present specificationby reference to the same extent as in a case where each of thedocuments, patent applications, technical standards is specifically andindividually indicated to be incorporated by reference.

The following appendices are further disclosed with respect to the aboveembodiments.

APPENDIX 1

An imaging apparatus (10, 200, 300, 400, 500, and 700) including animaging element (44, 244, 344, 444, and 544), the imaging apparatus (10,200, 300, 400, 500, and 700) comprising a storage portion (64) thatstores captured image data (70 and 570) obtained by imaging a subject bythe imaging element (44, 244, 344, 444, and 544) and is incorporated inthe imaging element (44, 244, 344, 444, and 544), a processing portion(62, 262, 363, 462, and 562) that processes the captured image data (70and 570) and is incorporated in the imaging element (44, 244, 344, 444,and 544), an output portion (62D, 262D, 362D, 462D, and 562D) thatoutputs processed image data obtained by processing the captured imagedata (70 and 570) by the processing portion (62, 262, 363, 462, and 562)and is incorporated in the imaging element (44, 244, 344, 444, and 544),and a plurality of signal processing portions (50, 250, 350, 450, 550,650, 52, 252, and 452) that are disposed outside the imaging element, inwhich the processing portion (62, 262, 362, 462, and 562) performsprocessing of dividing the captured image data (70 and 570) stored inthe storage portion (64) into a plurality of pieces of image data (70Aand 70B), the output portion (62D, 262D, 362D, 462D, and 562D) includesa plurality of output lines (53 and 55) each disposed in correspondencewith each of the plurality of signal processing portions (50, 250, 350,450, 550, 650, 52, 252, and 452) and outputs each of the plurality ofpieces of image data (70A and 70B) as the processed image data to acorresponding signal processing portion among the plurality of signalprocessing portions (50, 250, 350, 450, 550, 650, 52, 252, 452, 750, and752) from the plurality of output lines (53 and 55), and any of theplurality of signal processing portions (50, 250, 350, 450, 550, 650,52, 252, 452, 750, and 752) combines the plurality of pieces of imagedata (70A and 70B).

APPENDIX 2

The imaging apparatus (200) according to Appendix 1, in which each ofthe plurality of pieces of image data (70A and 70B) is image dataindicating an image having an overlapping region (71 and 271) betweenadjacent images (70A1 a, 70B1 a, 70A2 a, and 70B2 a) among a pluralityof images (70A1 a, 70B1 a, 70A2 a, and 70B2 a) based on each of theplurality of pieces of image data (70A and 70B).

APPENDIX 3

The imaging apparatus (200) according to Appendix 1 or 2, in which theplurality of images (70A2 a and 70B2 a) are divided into a designatedimage (70B2 a) and an image different from the designated image.

APPENDIX 4

The imaging apparatus (200) according to Appendix 3, in which theprocessing portion detects face image data indicating an image (69) of aface from the captured image data (70), and the designated image (70B2a) is an image including the image (69) of the face indicated by theface image data detected by the processing portion (262) in a capturedimage indicated by the captured image data (70).

APPENDIX 5

The imaging apparatus (400) according to any one of Appendices 1 to 4,in which the processing portion (462) changes a division method for thecaptured image data between a recording imaging mode and a displaymotion picture capturing mode.

APPENDIX 6

The imaging apparatus according to Appendix 5, in which the processingportion (462) divides the captured image data (70) into a plurality ofpieces of overlapping image data (70A1 and 70B1) as the plurality ofpieces of image data in the recording imaging mode, and divides thecaptured image data (70) in units of lines in the display motion picturecapturing mode, and each of the plurality of pieces of overlapping imagedata is image data indicating an image having an overlapping region (71)between adjacent images among a plurality of images.

What is claimed is:
 1. An image sensor comprising: a memory that storescaptured image data obtained by imaging a subject via the image sensorand that is incorporated in the image sensor; and a processor that isincorporated in the image sensor, wherein: the processor includes anoutput circuit, and the output circuit includes a plurality of outputlines and outputs each of a plurality of pieces of image data into whichthe captured image data stored in the memory is divided, from theplurality of output lines.
 2. The image sensor according to claim 1,wherein each of the plurality of pieces of image data is image dataindicating an image having an overlapping region between adjacent imagesamong images based on each of the plurality of pieces of image data. 3.The image sensor according to claim 1, wherein a plurality of images aredivided into a designated image and an image different from thedesignated image.
 4. The image sensor according to claim 3, furthercomprising: a detection processor configured to detect face image dataindicating an image of a face from the captured image data, wherein thedesignated image is an image including the image of the face indicatedby the face image data detected by the detection processor in a capturedimage indicated by the captured image data.
 5. The image sensoraccording to claim 1, wherein a method of dividing the captured imagedata varies between a recording imaging mode and a display movingpicture capturing mode.
 6. The image sensor according to claim 5,wherein the captured image data is divided into a plurality of pieces ofoverlapping image data as the plurality of pieces of image data in therecording imaging mode, and the captured image data is divided intounits of lines in the display moving picture capturing mode.
 7. Theimage sensor according to claim 6, wherein each of the plurality ofpieces of overlapping image data is image data indicating an imagehaving an overlapping region between adjacent images among a pluralityof images.
 8. The image sensor according to claim 5, wherein therecording imaging mode is an operation mode in which the imaging elementperforms imaging for a still picture.
 9. The image sensor according toclaim 1, wherein the captured image data is color image data indicatinga color captured image in which a plurality of primary color pixels areperiodically arranged, the color image data is divided into a pluralityof pieces of primary color pixel arrangement image data as the pluralityof pieces of image data, and each of the plurality of pieces of primarycolor pixel arrangement image data is image data indicating an image inwhich each of the plurality of primary color pixels is periodicallyarranged.
 10. The image sensor according to claim 9, wherein theplurality of pieces of primary color pixel arrangement image data are aplurality of pieces of divided image data obtained by thinning out andthen dividing the color image data.
 11. The image sensor according toclaim 10, wherein the plurality of pieces of divided image data areodd-numbered column image data indicating a set of pixels ofodd-numbered columns and even-numbered column image data indicating aset of pixels of even-numbered columns, in thinned image data obtainedby thinning out the color image data in units of rows.
 12. The imagesensor according to claim 1, wherein the plurality of pieces of imagedata are a plurality of pieces of compressed image data obtained bycompressing the captured image data by dividing the captured image datainto a plurality of bit ranges.
 13. The image sensor according to claim12, wherein the plurality of pieces of compressed image data arehigh-order bit image data and low-order bit image data in the capturedimage data.
 14. The image sensor according to claim 1, wherein, in theimage sensor, at least a photoelectric conversion element and the memoryare formed in one chip.
 15. The image sensor according to claim 14,wherein the image sensor is a laminated image sensor in which thephotoelectric conversion element is laminated with the memory.
 16. Animage data processing method of an image sensor including a memory thatstores captured image data obtained by imaging a subject, and aprocessor, wherein the memory and the processor are incorporated in theimage sensor, the image data processing method comprising: outputtingeach of a plurality of pieces of image data into which the capturedimage data stored in the memory is divided, from a plurality of outputlines, wherein: the processor includes an output circuit, and the outputcircuit includes the plurality of output lines.
 17. A non-transitorycomputer-readable storage medium storing a program causing a computerapplied to an image sensor including a memory that stores captured imagedata obtained by imaging a subject, and a processor, wherein the memoryand the processor incorporated in the image sensor incorporates anoutput circuit, the processor includes an output circuit, and the outputcircuit includes a plurality of output lines, to execute a processcomprising: outputting each of a plurality of pieces of image data intowhich the captured image data stored in the memory is divided, from theplurality of output lines.